Frontend

Overview

The Dashboard is a React 19 application built with TanStack ecosystem (Router, DB, Table) and shadcn/ui components. It uses TanStack DB for reactive client-side storage with automatic localStorage persistence, providing real-time visualization of restaurant comment sentiment analysis with cross-tab synchronization.

Purpose

Display live comment stream via SSE
Show aggregate statistics and charts
Provide paginated table with filtering and search
Real-time updates without page refresh
Persist comments locally with cross-tab synchronization
Responsive UI with Tailwind CSS

Architecture

┌─────────────────────────┐
│  API (HTTP + SSE)       │
│  localhost:3001         │
└───────────┬─────────────┘
            │
       ┌────┴─────┐
       │ SSE      │ Batch Load (Initial)
       │          │
       ▼          ▼
┌──────────┐  ┌──────────────────┐
│ useSSE   │  │ useCommentsData  │
│ Hook     │  │ Loader           │
└────┬─────┘  └──────┬───────────┘
     │               │
     │               ▼
     │      ┌─────────────────────────┐
     └─────>│  TanStack DB Collection │
            │  (localStorage backend) │
            │  - Schema validation    │
            │  - Auto persistence     │
            │  - Cross-tab sync       │
            └──────────┬──────────────┘
                       │
                       ▼
            ┌─────────────────────┐
            │  useLiveQuery       │
            │  (Reactive queries) │
            └──────────┬──────────┘
                       │
                       ▼
            ┌──────────────────────────┐
            │  React Components        │
            │  - CommentsTable         │
            │  - StatisticsCharts      │
            │  - Home/Comments pages   │
            └──────────────────────────┘

Technology Stack

TanStack DB

Purpose: Client-side database with reactive queries and local persistence

Why TanStack DB:

Built-in localStorage persistence
Automatic cross-tab synchronization
Reactive live queries (components auto-update)
Schema validation with Zod
Type-safe queries similar to SQL
No manual state management needed

Collection Setup:

// dashboard/src/lib/comments-collection.ts
import { createCollection } from '@tanstack/react-db'
import { localStorageCollectionOptions } from '@tanstack/react-db'
import { z } from 'zod'
 
// Define schema with validation and transformations
export const processedCommentSchema = z.object({
  id: z.number(),
  commentId: z.string(),
  text: z.string(),
  textHash: z.string(),
  tag: z.enum(['positive', 'negative', 'neutral', 'unrelated']),
  processedAt: z.union([z.string(), z.date()])
    .transform(val => typeof val === 'string' ? new Date(val) : val),
  consumerId: z.string(),
  source: z.string(),
  retryCount: z.number(),
})
 
// Create collection with localStorage backend
export const commentsCollection = createCollection(
  localStorageCollectionOptions({
    id: 'processed-comments',
    storageKey: 'kafka-dashboard-comments',
    getKey: (item) => item.commentId,
    schema: processedCommentSchema,
  })
)

Key Features:

Schema Validation: All data validated with Zod before insertion
Type Transformations: Automatically converts processedAt strings to Date objects
Deduplication: Uses commentId as unique key
Persistence: All changes auto-saved to localStorage
Cross-tab Sync: Updates in one tab appear in all tabs instantly

Usage in Components:

import { useLiveQuery, eq } from '@tanstack/react-db'
import { commentsCollection } from '#/lib/comments-collection'
 
function CommentsTable() {
  // Get all comments, sorted by newest first
  const { data: comments = [] } = useLiveQuery((q) => 
    q
      .from({ comment: commentsCollection })
      .orderBy(({ comment }) => comment.processedAt, 'desc')
  )
 
  // Filter by tag
  const { data: positiveComments = [] } = useLiveQuery((q) => 
    q
      .from({ comment: commentsCollection })
      .where(({ comment }) => eq(comment.tag, 'positive'))
  )
 
  // Component automatically re-renders when data changes
  return <div>{comments.length} comments</div>
}

Inserting Data:

// From SSE
commentsCollection.insert({
  id: 1,
  commentId: 'abc123',
  text: 'Great food!',
  tag: 'positive',
  processedAt: '2024-01-01T00:00:00Z', // Auto-converted to Date
  consumerId: 'consumer-1',
  source: 'twitter',
  retryCount: 0,
})
 
// Batch insert during initial load
comments.forEach(comment => {
  commentsCollection.insert(comment)
})

Data Flow:

Initial Load: API → Batch loader → Collection → localStorage
SSE Updates: SSE → Schema validation → Collection → localStorage → Live queries
Component Access: Live query → Collection → Components (reactive)
Cross-tab: Tab 1 insert → localStorage → Tab 2 storage event → Collection update

TanStack Router

Purpose: Type-safe routing with file-based routes

Route Structure:

dashboard/src/routes/
├── __root.tsx        # Root layout with Header/Footer
├── index.tsx         # Home page (statistics + charts)
└── comments.tsx      # Comments page (table)

Root Layout:

// __root.tsx
export const Route = createRootRoute({
  component: RootLayout
})
 
function RootLayout() {
  return (
    <>
      <Header />
      <Outlet />  {/* Child routes render here */}
      <Footer />
    </>
  )
}

No manual route configuration needed - file structure defines routes

Initial Data Loading

Purpose: Batch load all comments from API into TanStack DB collection

Hook: dashboard/src/hooks/useCommentsDataLoader.ts

import { commentsApi } from '#/lib/api'
import { commentsCollection } from '#/lib/comments-collection'
 
const CHUNK_SIZE = 100 // Load 100 comments per chunk
 
export function useCommentsDataLoader() {
  const [progress, setProgress] = useState({
    isLoading: false,
    loaded: 0,
    total: 0,
    error: null,
  })
 
  useEffect(() => {
    async function loadCommentsInBatches() {
      // Get first chunk to determine total
      const firstChunk = await commentsApi.getComments({
        page: 1,
        pageSize: CHUNK_SIZE,
      })
 
      // Insert into collection (auto-persists to localStorage)
      firstChunk.data.forEach(comment => {
        commentsCollection.insert(comment)
      })
 
      // Load remaining chunks in parallel (max 3 concurrent)
      const totalPages = firstChunk.totalPages
      const maxConcurrent = 3
      
      for (let i = 2; i <= totalPages; i += maxConcurrent) {
        const chunks = await Promise.all(/* ... */)
        
        chunks.forEach(chunk => {
          chunk.data.forEach(comment => {
            commentsCollection.insert(comment)
          })
        })
        
        setProgress({ loaded, total, isLoading: true })
      }
    }
 
    loadCommentsInBatches()
  }, [])
 
  return progress
}

Why Batch Loading:

Efficient API usage (fewer requests)
Progress tracking for UX
Data immediately available in collection
Persisted to localStorage for offline access
Only runs once per session

Subsequent Visits:

Comments load instantly from localStorage
No API call needed
SSE keeps data fresh

TanStack Table

Purpose: Headless table library for complex data grids

Location: dashboard/src/components/CommentsTable.tsx

Implementation:

import {
  useReactTable,
  getCoreRowModel,
  getPaginationRowModel,
  getSortedRowModel,
  getFilteredRowModel,
  flexRender,
} from '@tanstack/react-table'
 
export function CommentsTable() {
  // State management
  const [sorting, setSorting] = useState<SortingState>([])
  const [columnFilters, setColumnFilters] = useState<ColumnFiltersState>([])
  const [currentPage, setCurrentPage] = useState(1)
 
  // Column definitions
  const columns = useMemo<ColumnDef<ProcessedComment>[]>(() => [
    {
      accessorKey: 'commentId',
      header: 'ID',
      cell: ({ row }) => (
        <div className="font-mono text-xs">
          {row.original.commentId.substring(0, 8)}...
        </div>
      ),
    },
    {
      accessorKey: 'text',
      header: 'Comment',
      cell: ({ row }) => (
        <div className="max-w-md truncate">
          {row.original.text}
        </div>
      ),
    },
    {
      accessorKey: 'tag',
      header: 'Sentiment',
      cell: ({ row }) => {
        const tag = row.original.tag
        const variantMap: Record<string, any> = {
          positive: 'default',
          negative: 'destructive',
          neutral: 'secondary',
          unrelated: 'outline',
        }
        return (
          <Badge variant={variantMap[tag]} className="capitalize">
            {tag}
          </Badge>
        )
      },
    },
    {
      accessorKey: 'source',
      header: 'Source',
      cell: ({ row }) => (
        <div className="capitalize">{row.original.source}</div>
      ),
    },
    {
      accessorKey: 'processedAt',
      header: 'Processed At',
      cell: ({ row }) => (
        <div className="text-xs text-muted-foreground">
          {new Date(row.original.processedAt).toLocaleString()}
        </div>
      ),
    },
    {
      accessorKey: 'retryCount',
      header: 'Retries',
      cell: ({ row }) => (
        <div className="text-center">
          {row.original.retryCount || 0}
        </div>
      ),
    },
  ], [])
 
  // Initialize table
  const table = useReactTable({
    data: data?.data || [],
    columns,
    getCoreRowModel: getCoreRowModel(),
    getPaginationRowModel: getPaginationRowModel(),
    getSortedRowModel: getSortedRowModel(),
    getFilteredRowModel: getFilteredRowModel(),
    state: {
      sorting,
      columnFilters,
    },
    onSortingChange: setSorting,
    onColumnFiltersChange: setColumnFilters,
  })
 
  return (
    <Table>
      <TableHeader>
        {table.getHeaderGroups().map((headerGroup) => (
          <TableRow key={headerGroup.id}>
            {headerGroup.headers.map((header) => (
              <TableHead key={header.id}>
                {flexRender(
                  header.column.columnDef.header,
                  header.getContext()
                )}
              </TableHead>
            ))}
          </TableRow>
        ))}
      </TableHeader>
      <TableBody>
        {table.getRowModel().rows.map((row) => (
          <TableRow key={row.id}>
            {row.getVisibleCells().map((cell) => (
              <TableCell key={cell.id}>
                {flexRender(
                  cell.column.columnDef.cell,
                  cell.getContext()
                )}
              </TableCell>
            ))}
          </TableRow>
        ))}
      </TableBody>
    </Table>
  )
}

Features:

Column sorting (click headers)
Filtering by tag dropdown
Text search across comments
Client-side pagination
Responsive design

State Management

TanStack DB Collections (Primary)

For Comments: Using TanStack DB with localStorage backend (see TanStack DB section above)

Why TanStack DB for Comments:

Automatic persistence to localStorage
Cross-tab synchronization built-in
Reactive queries (no manual subscriptions)
Schema validation ensures data quality
SQL-like query syntax
No memory limits (uses browser storage)
Data survives page refresh
Offline support

Query Examples:

// Get all comments
const { data: allComments = [] } = useLiveQuery((q) => 
  q.from({ comment: commentsCollection })
)
 
// Filter by tag
const { data: positiveComments = [] } = useLiveQuery((q) => 
  q
    .from({ comment: commentsCollection })
    .where(({ comment }) => eq(comment.tag, 'positive'))
, [])
 
// Sort by date
const { data: recentComments = [] } = useLiveQuery((q) => 
  q
    .from({ comment: commentsCollection })
    .orderBy(({ comment }) => comment.processedAt, 'desc')
    .limit(20)
)
 
// Complex query
const { data: filteredComments = [] } = useLiveQuery((q) => {
  let query = q.from({ comment: commentsCollection })
  
  if (tagFilter) {
    query = query.where(({ comment }) => eq(comment.tag, tagFilter))
  }
  
  return query.orderBy(({ comment }) => comment.processedAt, 'desc')
}, [tagFilter])

Benefits of TanStack DB:

No manual state updates needed
Automatic persistence to localStorage
Query-based access with SQL-like syntax
Better for large datasets
Type-safe queries with TypeScript
Cross-tab synchronization built-in
Reactive updates without manual subscriptions

Statistics Computation

Computed from TanStack DB Collection:

Statistics are computed on-the-fly from the comments collection rather than stored separately. This ensures data consistency and eliminates synchronization issues.

import { useLiveQuery, eq } from '@tanstack/react-db'
 
// Get all comments for statistics
const { data: allComments = [] } = useLiveQuery((q) => 
  q.from({ comment: commentsCollection })
)
 
// Compute statistics in real-time
const statistics = useMemo(() => {
  const hourAgo = new Date(Date.now() - 60 * 60 * 1000)
  
  return {
    total: allComments.length,
    byTag: {
      positive: allComments.filter(c => c.tag === 'positive').length,
      negative: allComments.filter(c => c.tag === 'negative').length,
      neutral: allComments.filter(c => c.tag === 'neutral').length,
      unrelated: allComments.filter(c => c.tag === 'unrelated').length,
    },
    recentCount: allComments.filter(c => c.processedAt >= hourAgo).length,
  }
}, [allComments])

Why compute instead of store:

Always accurate (no sync issues)
Single source of truth (comments collection)
Automatic updates when collection changes
No separate state management needed
Simpler architecture

Server-Sent Events (SSE)

Hook: dashboard/src/hooks/useSSE.ts

Purpose: Establishes SSE connection to API and inserts incoming comments into TanStack DB collection

import { useEffect, useRef, useState } from 'react'
import { SSE_URL } from '../lib/api'
import { commentsCollection, processedCommentSchema } from '../lib/comments-collection'
 
export function useSSE() {
  const [isConnected, setIsConnected] = useState(false)
  const [error, setError] = useState<string | null>(null)
  const eventSourceRef = useRef<EventSource | null>(null)
 
  useEffect(() => {
    const eventSource = new EventSource(SSE_URL)
    eventSourceRef.current = eventSource
 
    eventSource.onopen = () => {
      console.log('SSE connection opened')
      setIsConnected(true)
      setError(null)
    }
 
    eventSource.onerror = (err) => {
      console.error('SSE error:', err)
      setIsConnected(false)
      setError('Connection lost. Retrying...')
    }
 
    // Listen to 'comment' events
    eventSource.addEventListener('comment', (event: MessageEvent) => {
      try {
        const rawComment = JSON.parse(event.data)
        // Validate and transform with schema
        const comment = processedCommentSchema.parse(rawComment)
        // Insert into collection (auto-persists to localStorage)
        commentsCollection.insert(comment)
        // Components using useLiveQuery automatically update
      } catch (err) {
        console.error('Error parsing/validating comment:', err)
      }
    })
 
    return () => {
      eventSource.close()
      setIsConnected(false)
    }
  }, [])
 
  return { isConnected, error }
}

Event Handling:

Comment events: Validated with Zod schema, inserted into TanStack DB collection
Error handling: Auto-reconnection handled by browser's EventSource

Data Flow:

SSE Comment Event → JSON Parse → Zod Validation → 
Collection Insert → localStorage Persist → 
Live Query Update → Component Re-render

Usage in pages:

export default function HomePage() {
  const { isConnected } = useSSE()  // Starts SSE connection
  
  return (
    <div>
      {isConnected ? '🟢 Live' : '🔴 Disconnected'}
    </div>
  )
}

Auto-reconnect:

Browser's EventSource automatically reconnects on connection loss
No manual retry logic needed

Additional Features

Schema Validation

All incoming data from SSE is validated using Zod schemas before insertion into the TanStack DB collection.

Purpose:

Type safety at runtime
Data integrity enforcement
Automatic type transformations
Prevent invalid data from corrupting storage

Implementation: The processedCommentSchema defined in the collection setup validates:

Required fields (commentId, text, tag, etc.)
Enum validation for tags (only allows: positive, negative, neutral, unrelated)
Date transformations (converts ISO strings to Date objects)
Type coercion where appropriate

Error Handling:

try {
  const rawComment = JSON.parse(event.data)
  const comment = processedCommentSchema.parse(rawComment) // Throws if invalid
  commentsCollection.insert(comment)
} catch (err) {
  console.error('Validation failed:', err)
  // Invalid data is logged but not inserted
}

Benefits:

Catches API contract changes early
Prevents runtime errors from invalid data
Self-documenting data structure
TypeScript integration for autocomplete

Loading Progress

Purpose: Visual feedback during initial batch data loading

Features:

Progress bar showing percentage of comments loaded
Current count vs total count display
Loading states during concurrent chunk fetching

Display:

Loading comments... 250 / 1000 (25%)
[████████░░░░░░░░░░░░]

Implementation: The useCommentsDataLoader hook returns progress state:

{
  isLoading: boolean,
  loaded: number,     // Comments loaded so far
  total: number,      // Total comments to load
  error: string | null
}

UI Updates:

Progress bar fills as chunks load
Shows completion when loaded === total
Only visible during initial load (not on subsequent visits)
Hides once localStorage is populated

UI Components

StatisticsCharts

Location: dashboard/src/components/StatisticsCharts.tsx

Uses Recharts library:

import { PieChart, Pie, Cell, BarChart, Bar, XAxis, YAxis } from 'recharts'
 
export function StatisticsCharts({ statistics }: Props) {
  const pieData = [
    { name: 'Positive', value: statistics.byTag.positive, color: '#22c55e' },
    { name: 'Negative', value: statistics.byTag.negative, color: '#ef4444' },
    { name: 'Neutral', value: statistics.byTag.neutral, color: '#6b7280' },
    { name: 'Unrelated', value: statistics.byTag.unrelated, color: '#eab308' },
  ].filter(item => item.value > 0)
 
  return (
    <div className="grid gap-6 md:grid-cols-2">
      {/* Pie Chart */}
      <Card>
        <CardHeader>
          <CardTitle>Sentiment Distribution</CardTitle>
        </CardHeader>
        <CardContent>
          <PieChart width={400} height={300}>
            <Pie
              data={pieData}
              label={({ name, percent }) => 
                `${name}: ${(percent * 100).toFixed(0)}%`
              }
              outerRadius={80}
              dataKey="value"
            >
              {pieData.map((entry, index) => (
                <Cell key={index} fill={entry.color} />
              ))}
            </Pie>
          </PieChart>
        </CardContent>
      </Card>
 
      {/* Bar Chart */}
      <Card>
        <BarChart data={barData} width={400} height={300}>
          <XAxis dataKey="tag" />
          <YAxis />
          <Bar dataKey="count" fill="#4f8fb8" />
        </BarChart>
      </Card>
    </div>
  )
}

Chart color scheme:

Positive: Green (#22c55e)
Negative: Red (#ef4444)
Neutral: Gray (#6b7280)
Unrelated: Yellow (#eab308)

Location: dashboard/src/components/StatisticsCharts.tsx

const COLORS = {
  positive: '#22c55e',
  negative: '#ef4444',
  neutral: '#6b7280',
  unrelated: '#eab308',
}

Badge Implementation

Location: dashboard/src/components/CommentsTable.tsx

Sentiment tags are displayed using standard shadcn/ui badge variants:

{
  accessorKey: 'tag',
  header: 'Sentiment',
  cell: ({ row }) => {
    const tag = row.original.tag
    const variantMap: Record<string, any> = {
      positive: 'default',      // Primary color badge
      negative: 'destructive',  // Red badge
      neutral: 'secondary',     // Gray badge
      unrelated: 'outline',     // Bordered badge
    }
    return (
      <Badge variant={variantMap[tag]} className="capitalize">
        {tag}
      </Badge>
    )
  },
}

Standard shadcn/ui variants used:

default - Primary color with filled background
destructive - Red background for negative
secondary - Gray background for neutral
outline - Bordered only for unrelated

State Management Architecture

TanStack DB (All Data):

Comments stored in TanStack DB collection
Statistics computed from comments collection
All queries use useLiveQuery for reactivity
localStorage persistence for offline support
Cross-tab synchronization built-in

Page Structure

Home Page (index.tsx)

Route: /

Features:

Statistics cards (Total, Positive %, Negative %, Last Hour)
Connection status indicator
Pie and bar charts
Real-time updates via TanStack DB reactivity

Data Flow:

Comments loaded from TanStack DB collection via useLiveQuery
Statistics computed from comments using useMemo
Auto-updates when new comments arrive via SSE
No separate API calls for statistics needed

Comments Page (comments.tsx)

Route: /comments

Features:

Search input (filters comment text)
Tag filter buttons (All, Positive, Negative, Neutral, Unrelated)
Sortable columns (click headers)
Real-time updates as new comments arrive
Pagination controls

Data Flow:

const { data: filteredByTag = [] } = useLiveQuery((q) => {
  let query = q.from({ comment: commentsCollection })
  
  if (tagFilter) {
    query = query.where(({ comment }) => eq(comment.tag, tagFilter))
  }
  
  return query.orderBy(({ comment }) => comment.processedAt, 'desc')
}, [tagFilter])
 
// Client-side search filtering (TanStack DB doesn't support full-text search)
const filteredComments = useMemo(() => {
  let filtered = tagFilter ? filteredByTag : allCommentsArray
 
  if (searchQuery) {
    const query = searchQuery.toLowerCase()
    filtered = filtered.filter(c => 
      c.text.toLowerCase().includes(query) ||
      c.commentId.toLowerCase().includes(query)
    )
  }
 
  return filtered
}, [allCommentsArray, filteredByTag, tagFilter, searchQuery])

Benefits:

No API calls for filtering (data already local)
Instant filtering with no network lag
Tag filtering uses reactive live queries
Search filtering done client-side
Table updates automatically when new comments arrive via SSE
All comments persisted to localStorage

Build & Development

Development Mode

cd dashboard
pnpm dev
# Runs on http://localhost:3000

Features:

Hot module replacement
Fast refresh
Vite dev server

Production Build

pnpm build
# Outputs to dashboard/dist/

Docker:

FROM nginx:alpine
COPY dist/ /usr/share/nginx/html
EXPOSE 3000

Styling

Tailwind CSS 4:

Utility-first classes
Dark modeDB localStorage caching (zero API calls after initial load)
Reactive queries only re-render affected components
Cross-tab sync via storage events (no polling)
Custom theme colors

Example:

<div className="grid gap-6 md:grid-cols-2 lg:grid-cols-4">
  <Card className="border-blue-200 dark:border-blue-800">
    <CardHeader className="pb-2">
      <CardTitle className="text-sm font-medium text-muted-foreground">
        Total Comments
      </CardTitle>
    </CardHeader>
  </Card>
</div>

Performance

Optimizations:

Memoized column definitions (useMemo)
Virtual scrolling not needed (pagination limits rows)
SSE more efficient than polling
TanStack Query caching reduces API calls

Bundle size: ~500KB (gzipped)

Testing

Manual Testing

# Start dashboard
pnpm dev
 
# Visit pages
open http://localhost:3000         # Home
open http://localhost:3000/comments # Comments table
 
# Check console
# Should see: "SSE connection opened"

Verify Real-time Updates

Open dashboard
Check "Live Stream Active" indicator
Navigate to /comments
Watch table update as new comments arrive
Check statistics on home page updating

Next Steps

API & SSE - Backend that serves the data
Setup Guide - Complete technology stack explanation