Back to projects
data 2025

Litoscope Rust WASM PWA

A research-analysis PWA case study that uses Rust compiled to WebAssembly for browser-side academic and policy data processing, with protected React workflows, offline persistence, and exports.

Open Litoscope
Technical focus
Rust/WASM browser data processing and PWA architecture perspective
Year
2025
Stack
Rust, WebAssembly, wasm-bindgen, React 19, TypeScript, Vite, PWA, Supabase Auth, Dexie, IndexedDB, Material React Table

Architecture

Browser-side research processing architecture

Litoscope is intentionally browser-heavy: React owns protected workflows and table UX, Rust compiled to WebAssembly performs the data transformations, and IndexedDB keeps processed research outputs local to the user's session.

  1. Stage 1

    PWA shell

    The application is a static Vite PWA with protected routes and installable app behavior.

    • React dashboard

      • React
      • Vite
      • Supabase
    • PWA runtime

      • Vite PWA
      • Service worker
    Technical details
    React dashboard
    Protected routing, Supabase-backed auth state, dashboard navigation, Academic and Policy workspaces, and route-aware theming.
    PWA runtime
    Service worker auto-update, WASM asset caching, installable manifest, and network-first caching for Supabase calls.
  2. Stage 2

    Input adapters

    Source files are parsed in the browser before data is sent into the WASM module.

    • Academic file input

      • TXT
      • File API
    • Policy workbook input

      • SheetJS
      • XLSX
    Technical details
    Academic file input
    Tab-delimited source exports are validated, parsed, and shaped for ecosystem mapping, corpus, and augmented-reference outputs.
    Policy workbook input
    Excel workbooks are read with SheetJS, converted to JSON sheet structures, then passed to Rust for policy publication and citation processing.
  3. Stage 3

    Rust and WASM core

    Heavy research transformations run locally through wasm-bindgen exports.

    • Academic processors

      • Rust
      • wasm-bindgen
    • Policy processors

      • Rust
      • serde
      • regex
    Technical details
    Academic processors
    Rust parses tab-delimited data, computes file stats, builds ecosystem rows, formats citations, extracts in-text citations, and calculates citation-per-year metrics.
    Policy processors
    Rust detects workbook sheets, normalizes publications and citations, merges policy references, links matched snippets, and calculates policy usage metrics.
  4. Stage 4

    Local outputs

    Processed outputs stay available locally for repeat inspection and export.

    • Dexie and IndexedDB

      • Dexie
      • IndexedDB
    • Tables and exports

      • MRT
      • CSV
      • PDF
      • DOCX
    Technical details
    Dexie and IndexedDB
    Academic files, policy files, last-viewed timestamps, processed payloads, and Material React Table state are persisted in browser storage.
    Tables and exports
    Searchable, filterable Material React Table views support selected, filtered, or complete exports to CSV, Excel, PDF, and Word.

Key flows

  1. Academic and policy uploads to Input adapters

    Browser-side readers validate source formats and shape tabular data without sending research files to a backend queue.

  2. Input adapters to Rust and WASM core

    Parsed text or workbook JSON enters wasm-bindgen functions for deterministic research transformations.

  3. Rust and WASM core to Dexie and IndexedDB

    Processed outputs and table state are stored locally so users can return to prior analysis sessions.

  4. Local outputs to Tables and exports

    Saved rows flow into searchable tables and controlled export paths for downstream academic and policy work.

Outcomes

Impact

  • Built the full Litoscope Academic and Litoscope Policy workflows for processing exported data source files and policy Excel workbooks directly in the browser.
  • Implemented Rust/WASM data pipelines for parsing, citation shaping, policy-publication matching, usage metrics, date normalization, and generated intermediate research outputs.
  • Documented a reproducible build path for compiling the Rust/WASM module, building the Vite PWA, and serving the static production bundle without exposing deployment-specific configuration.

Litoscope is a research-analysis PWA for turning academic and policy source files into structured intermediate outputs. The application has two major product areas: Litoscope Academic for exported data source files and Litoscope Policy for Excel workbooks from policy-document workflows.

The project combines a React and TypeScript interface with a Rust module compiled to WebAssembly. The browser handles authentication, uploads, tables, persistence, and exports, while Rust performs the heavier parsing and transformation work locally in the client runtime.

Product Scope

Litoscope Academic helps users process tab-delimited .txt files from exported data sources. It produces separate views for ecosystem mapping, corpus analysis, and augmented references.

Litoscope Policy supports .xlsx and .xls uploads. The frontend reads the workbook with SheetJS, converts sheets into JSON, and sends that data to the Rust/WASM module for policy-publication and citation processing.

Both flows share the same high-level product behavior:

  • upload a source file
  • validate and parse the format
  • process the source into structured research outputs
  • store processed results locally
  • inspect results in searchable, filterable tables
  • export filtered, selected, or complete table data

Rust and WebAssembly Pipeline

The Rust module exposes browser-callable functions through wasm-bindgen. The module handles both general file-processing utilities and domain-specific research transformations.

The academic pipeline includes:

  • tab-delimited file parsing
  • file statistics calculation
  • ecosystem mapping from exported data source fields
  • corpus generation with formatted citations and in-text citation extraction
  • augmented-reference generation
  • citation-per-year style metrics using publication year and citation fields

The policy pipeline includes:

  • Excel-derived JSON workbook processing
  • flexible sheet detection for results and matched references
  • publication and citation normalization
  • citation linking back to policy publications
  • policy snippet and matched-snippet handling
  • source/date/topic/SDG metadata shaping
  • policy reference formatting for exportable outputs

This keeps the data-heavy transformations close to the browser session and avoids sending research files through a backend processing queue.

Browser Data Layer

The app is designed around local continuity. Processed files and table state are stored in IndexedDB through Dexie, with separate tables for academic files, policy files, and persisted Material React Table state.

That storage layer lets users return to previously processed files, preserve table filters and sorting, track last-viewed timestamps, and delete local processed data when it is no longer needed.

The frontend also uses a cached WASM loader hook so the generated module is initialized once and reused across the app instead of being reloaded by each feature area.

Frontend Experience

The protected dashboard is built with React Router and Supabase Auth. Users move between the dashboard, Litoscope Academic, and Litoscope Policy through a collapsible sidebar, with route-based theming for the academic and policy sections.

The table layer uses Material React Table and TanStack Table behavior for search, filtering, sorting, row selection, pagination, and virtualization on larger datasets. Export flows support CSV, Excel, PDF, and Word output, with scope controls for all rows, selected rows, or the currently filtered table.

The PWA setup uses Vite PWA with auto-update behavior, asset caching, WASM file inclusion, an installable manifest, and network-first caching for Supabase API calls.

Build and Release Shape

The production build is shaped around a reproducible Rust/WASM and Vite sequence. The build compiles the Rust module first, writes generated WASM bindings into the frontend source tree, then builds the Vite frontend.

The containerized build path installs the Node and Rust toolchains, builds the WASM and frontend artifacts, and serves the static output. Public documentation avoids environment-variable names, hosting configuration, and internal deployment wiring.

The important technical point is reproducibility: the Rust/WASM toolchain is explicit, and the browser application can be rebuilt without relying on manual local steps.

Technical Perspective

This case study demonstrates browser-heavy product engineering with a non-trivial Rust/WASM core. The visible scope covers frontend application architecture, WebAssembly integration, local database design, research data transformation, protected user flows, table UX, export generation, and production-readiness documentation.