ResumeLet's Talk →
Quant Finance·Data Engineering·Full-Stack·2026

Distributed Market Intelligence
& Self-Validating Quant Platform

A self-validating quant pipeline with live job tracking, a Next.js control plane, and a static Vercel demo for recruiter walkthroughs.

Most projects stop at results — this one validates them in real time, streams every stage over WebSocket, ships signals and backtests, and exports a fixture-backed demo that runs without FastAPI or PySpark.

Next.jsFastAPIPySparkDuckDBWebSocketQADemo / Vercel
View repository →
01 — The Problem

Four gaps most financial projects ignore.

Most projects stop at "I got results." These four gaps matter in production.

01
Scalability
Pandas breaks at scale — PySpark handles multi-ticker workloads.
02
Correctness
Rolling metrics and drawdown need careful window functions per ticker.
03
Validation
Formal QA scores Spark output against a Pandas/SciPy benchmark.
04
Observability
WebSocket streams every pipeline stage — no black-box runs.
02 — How It Works

Four layers, seven pipeline stages — plus a static demo.

Layer 01UI Layer

Next.js live dashboard (web/) + static portfolio demo (demo/) sharing components

Layer 02API Layer

FastAPI — POST /run-pipeline, WebSocket /ws/{job_id}, /api/v1/*

Layer 03Processing Layer

PySpark — ingest, validate, features, QA, signals, backtest, versioned export

Layer 04Job & Persistence

BackgroundTasks + job_id + DuckDB history; single active-run lock (409)

Core pipeline stages
  1. 1FETCHING_DATA
  2. 2VALIDATION
  3. 3FEATURE_ENGINEERING
  4. 4RUNNING_QA
  5. 5SIGNAL_GENERATION
  6. 6BACKTEST
  7. 7REPORT_EXPORT

Then SAVING_DATA → COMPLETED; UI fetches via /features, /qa, and /api/v1/*. Live stack: API :8080 + web :3002. Demo stack: demo/ :3003 on fixtures only.

03 — Highlights

What ships today.

✓
Key Differentiator
QA Validation System
Spark outputs benchmarked against Pandas/SciPy — per-ticker alignment % in qa_report.json (>95% target).
📊
Signals & Backtest
Rule-based BUY/SELL/HOLD with confidence + reasons; signal_rules_v1 metrics (CAGR, Sharpe, max DD, win rate).
🔴
Real-time Execution
WebSocket timeline for all stages; job_id routing; 409 lock on concurrent runs.
🖥
Next.js Control Plane
Live web/ dashboard — Jobs, Explorer, QA, Signals, Regime, Backtest, Reports, Settings.
🚀
Static Demo / Vercel
demo/ app with fixture snapshot, historical step replay, and refresh scripts — no API required to deploy.
⚙
Local Research Platform
backend/ + web/ + demo/ + config/; Docker Compose; start_live_dev.sh / start_demo_dev.sh.
04 — Status

Build status — all planned phases complete.

✓
Phase 1
PySpark pipeline — validation, features, QA, signals, backtest, versioning
✓
Phase 2
FastAPI backend & data serving layer (/api/v1)
✓
Phase 3
Next.js live dashboard (web/)
✓
Phase 4
WebSocket real-time timeline + job_id tracking
✓
Phase 5
Static demo mode + Vercel deploy (demo/ fixtures)

Phases 1–5 are complete: PySpark pipeline, FastAPI, Next.js live UI, WebSocket job tracking, and the static Vercel demo with fixture capture. Optional research waves (richer indicators, HMM regimes, HTML/PDF reports, formal pytest goldens) remain outside the shipped core.

Contents
Distributed Market Intelligence & Quant QA System · 2026Completed — live stack + Vercel demo