Reproducibility Report

Dated test results.
Commit-pinned. Independently verifiable.

Every number below was produced by running the code. The commit, command, environment, and output are recorded. You can reproduce any result.

01 — Verified demo run

The canonical demo.
Run it yourself.

Fixture: examples/vendor-model-sample.csv (600 rows, SHA256: 79efd015a8501d1fa7ba79931532f5683b4c0e5f9bb56263d2544a8877cb3bac)

Commit
6259b1d (2026-10-23); results verified by running the command on 2026-11-04
Fixture
vendor-model-sample.csv (600 rows, SHA256: 79efd015a8501d1fa7ba79931532f5683b4c0e5f9bb56263d2544a8877cb3bac)
Command
npx tsx scripts/certify-model.ts examples/vendor-model-sample.csv --configurations 20
Recomputed Accuracy
69.33% (416 / 600 correct)
Claimed Accuracy
78.00% (ignored)
Wilson 95% CI
65.5% – 72.9%
Baselines
5 run, 5/5 beaten — best baseline: momentum-5d at 51.00%
Walk-Forward Folds
5 folds: 80.00% → 77.50% → 71.67% → 61.67% → 55.83%
Fall Across Folds
−24.17 pts
Leakage Gate
Passed
Verdict
OVERFIT · exit code 1
PBO
Null (no candidates file supplied)
Calibration
Ran (no confidence column in fixture)

Verification steps

  1. Clone the repository
  2. Verify fixture SHA256: sha256sum examples/vendor-model-sample.csv
  3. Run: npx tsx scripts/certify-model.ts examples/vendor-model-sample.csv --configurations 20
  4. Confirm output matches the table above exactly
02 — Certifier test suite

1,123 tests passing.
49/49 mutations caught.

Test results from pnpm test run at repo HEAD 6259b1d on 2026-11-04 (Node 22, Linux x64).

Test Files121

1,123 tests passing, 112 skipped

Vitest suite covering certifier gates, verdict logic, platform APIs, database integrity, and mutation verification.

  • Certifier: 45 files, ~600 tests (gates, verdict, edge cases)
  • Platform: 76 files, ~500 tests (API contracts, DB, auth, backtest)
  • Clean-slate: 8 files (visibility, repo hygiene)
    • Guard Mutations49/49 caught

    Each mutation reintroduces a known defect

    A guard that cannot fail a build is a report. Every mutation here was verified to fail its test when applied.

    • 35 certifier mutations (gate logic, verdict combining, edge cases)
    • 14 platform mutations (API contracts, DB integrity, auth)
    • 4 removed as vacuous (test asserted reported numbers, not gate identity)
    • Run: pnpm test -- --reporter=verbose to see mutation results
    Duration~5 minutes

    Full suite completes in under 5 minutes

    Node 22, Linux x64, 16GB RAM. No external services required for certifier tests.

    • Transform: 1.57s
    • Setup: 2.39s
    • Import: 13.92s
    • Tests: 442.68s
    • Environment: 7.28s

    Key test files

    Certifier CLI parity
    src/app/certify/cli-parity.spec.ts — CLI and /certify page produce identical verdicts
    Dashboard accuracy
    src/app/(app)/dashboard/dashboard-page.spec.tsx — recomputed accuracy matches engine
    Governance
    src/app/(app)/governance/governance-page.spec.tsx — audit trail integrity
    Investor page
    src/app/(app)/investor/investor-page.spec.tsx — portfolio risk surfaces
    Data page
    src/app/(app)/data/data-page.spec.tsx — market data pipeline
    Clean-slate visibility
    tests/clean-slate/visibility.spec.ts — repo privacy verified via GitHub API
    Certify PBO
    scripts/certify-pbo.spec.ts — 14.4% vs 40.0% README contrast reproduced
    TimesFM worker
    tests/ml/timesfm25-worker.spec.ts — quantile band correctness, model residency
    03 — Platform test suite

    API contracts, DB integrity,
    paper trading logic.

    Platform tests require PostgreSQL. Run pnpm exec prisma migrate deploy && pnpm test.

    API Routes Tested
    102/102 routes covered by integration tests
    Database Models
    50 Prisma models — CRUD, relations, cascades, constraints
    Paper Trading
    Order matching, fill simulation, cash flow, P&L attribution
    Forecasting Engine
    Generation, scoring, calibration, tournament, cross-sectional
    Backtest Builder
    Strategy definition, walk-forward, regime conditioning
    Auth / RBAC
    Clerk integration, role-based access, session management
    04 — CI pipeline

    27 blocking steps.
    Every PR must pass.

    GitHub Actions workflow. Matrix: Node 22, PostgreSQL 16. No merge without green.

    Step 01Typecheck

    tsc --noEmit — zero errors

    • Strict mode, no any without // @ts-expect-error
    • Run: pnpm run typecheck
    Step 02Lint

    eslint — zero warnings

    • Next.js config, React hooks, TypeScript ESLint
    • Run: pnpm run lint
    Step 03Test Suite

    1,123 tests — all must pass

    • Vitest with coverage thresholds
    • Mutation verification step
    Step 04Build

    next build — 73 pages, 102 API routes

    • Prisma generate included
    • Static pages verified
    Step 05Artifact Validation

    Proof pack rebuilt & validated

    • Byte-for-byte against source tree
    • CLIENT_PROOF_PACK.zip regenerated
    Step 06Repo Visibility

    GitHub API check — repo is private

    • tests/clean-slate/visibility.spec.ts
    • Fails if repo becomes public
    Workflow File
    .github/workflows/ci.yml
    Matrix
    Node 22.x, PostgreSQL 16, Ubuntu latest
    Secrets
    DATABASE_URL, GITHUB_TOKEN (no market data keys)
    Duration
    ~8–12 minutes typical
    05 — Independent verification

    No repository access needed.
    Run the demo package.

    The reproducibility package contains everything needed to verify the certifier without cloning the repo.

    Downloadable Assets

    Verification Steps (No Repo Needed)

    1. Download vendor-model-sample.csv
    2. Verify SHA256: 79efd015a8501d1fa7ba79931532f5683b4c0e5f9bb56263d2544a8877cb3bac
    3. Install Node 22
    4. Run: npx tsx scripts/certify-model.ts vendor-model-sample.csv --configurations 20
    5. Confirm output matches §1 exactly

    Browser vs CLI

    The homepage terminal demo runs the same algorithm in-browser (JavaScript port of the Wilson interval and fold logic in assets/js/app.js). The CLI is the authoritative implementation. Both produce identical results for the demo fixture.

    Every number is a receipt. The code is the proof.

    Request the full proof pack See technical evidence