Performance Benchmarks

Performance Benchmarks

Status: this fork's numbers are stale, and the harness itself needs a port pass

Every number below this section was measured against the base template's per-user-SQLite architecture (make prod.cluster, @bitclaw/loadtest), before the Postgres port. They are kept here as a methodology reference and postmortem , the lessons about cookie-signing bugs and how to spot an invalid "100% success" run are still directly useful , but do not quote any throughput/latency number below as current for this fork. The architecture they measured no longer exists here (shared Postgres instead of per-user SQLite files, withUserTransaction instead of withWriteLock).

Worse than just stale: the load-test harness itself is not fully ported and does not currently compile against this fork's schema. Confirmed by direct read:

  • scripts/seed-loadtest.ts still imports getUserDb from @/lib/db/user-db (removed , there are no per-user DB connections anymore) and randomUUIDv7 from 'bun' (Bun-only, not portable; should be uuidv7() from the uuidv7 package per architecture/postgres-port.md).
  • tests/load-test-admin.ts has the same randomUUIDv7 from 'bun' import.
  • Makefile's loadtest.seed target (and the e2e targets) still pass DATABASE_PATH/USER_DATA_DIR , dead env var names in this fork (see docs/warpkit/deployment.md, which uses DATABASE_URL).
  • Partially already ported, for contrast: src/routes/api/loadtest/auth.ts already correctly imports uuidv7 as randomUUIDv7 from the uuidv7 package , so this isn't a uniformly-untouched surface, just an inconsistently-finished one.

This matches CLAUDE.md's Postgres Port Status section, which separately flags src/routes/api/loadtest/ as one of the areas where a full bunx tsc --noEmit still fails , this doc's own investigation confirms the load-test harness specifically (not just that route file) is part of that same unfinished surface.

What re-benchmarking this fork correctly would require, not yet done:

  1. Port scripts/seed-loadtest.ts and tests/load-test-admin.ts off getUserDb/bun's randomUUIDv7, onto shared-Postgres inserts scoped by user_id and the uuidv7 package , same shape as any other ported feature.
  2. Re-derive the seeding strategy: the old harness created 1000 per-user SQLite files; the Postgres equivalent is 1000 rows across the shared users/sessions/etc. tables, no per-file step needed at all.
  3. Fix the Makefile targets' env vars to DATABASE_URL.
  4. Re-run against both deploy targets this fork actually has , self-hosted (make prod.cluster, comparable to the old methodology) and Cloudflare Workers (a fundamentally different cost model: no clustering/reusePort, Hyperdrive connection overhead per isolate, Workers' own request-level scaling instead of OS-level worker processes) , and report them as two separate result sets, not one.

Until that port pass happens, treat this fork as unbenchmarked. The sections below are preserved for their methodology value only.


(Historical, pre-port) Application-level load test results using @bitclaw/loadtest against a production build (make prod.cluster)

These are relative-comparison numbers (before/after an optimization on the same box), not absolute capacity guarantees, and they were never guarantees even for the architecture they measured. Read the caveats below before drawing any conclusion from them, and remember they describe per-user SQLite, not this fork's shared Postgres.

2026-07-19: this document was rewritten from scratch (pre-port)

Every number that ever appeared in this doc before that date measured the wrong thing. Two independent bugs in the load-test harness meant every "authenticated" request was either hitting a cold DB path or silently redirecting to /login and getting counted as a passing 200. Both were fixed at the time; the numbers below were the first trustworthy benchmark in the base template's history at that point. Full postmortem at the bottom, under What was wrong before today , worth reading once for the methodology lesson, regardless of which DB architecture you're benchmarking.

Known gap (pre-port, and still true here): deploy-overlap / job-queue-startup concurrency is not covered

Every run in this section exercises a single, already-running server via HTTP load. It never simulates a rolling zero-downtime deploy (new process starts and opens its DB handles before the old one stops , both briefly run at once) or job-queue startup racing another process. That's exactly the class of bug that hit @bitclaw/jobs's SQLite predecessor in production, root-caused to a SQLite pragma-ordering bug. This fork's Postgres-backed queue (FOR UPDATE SKIP LOCKED claims, see docs/warpkit/features/jobs.md) is architecturally less exposed to that specific bug class , concurrent claims are a normal Postgres pattern, not a SQLite-locking edge case , but that's an architectural argument, not a verified regression test. Not built yet; flagged here as a follow-up, same as before the port.

Test Setup (pre-port , will not run as-is, see Status above)

# Terminal 1 - start clustered production server
LOADTEST_AUTH_ENABLED=true LOADTEST_EMAIL=loadtest@example.com LOADTEST_OTP=loadtest123 make prod.cluster

# Terminal 2 - seed the 1000-user pool (must run under NODE_ENV=production - see below)
make loadtest.seed

# Terminal 2 - run load test
LOADTEST_EMAIL=loadtest@example.com LOADTEST_OTP=loadtest123 bun run test:load
bun run test:load:admin

make loadtest.seed forces NODE_ENV=production internally , seeded session cookies must be signed the same way the production server that will read them expects, or every authenticated request silently redirects to /login (see the postmortem below). This mechanism is still correct in shape; only the DB calls inside seed-loadtest.ts need porting (see Status above).


Test Environment Caveats (pre-port , still generally true, kept for methodology)

Read these before trusting any number once this harness is re-ported and re-run.

  • WSL2, not bare metal. All historical runs are on WSL2 (virtual disk + virtual network stack) on Windows. A real Linux VPS, or Cloudflare's actual edge for the Workers target, will produce different numbers.
  • Run-to-run variance is real and can be large if the host isn't idle. repeat: 3 (median + CoV%) was the mitigation , treat any delta below the reported CoV% as noise, and treat a run where the static landing page number is off from prior runs as unreliable regardless of what the other rows say (it's pure Bun.file() I/O on self-hosted, untouched by anything DB-related , if it moves, the host was busy, not the app).
  • Thresholds are application defaults (@bitclaw/loadtest): P95 ≤ 500ms, success ≥ 95%, throughput ≥ 50 req/s at lowest concurrency.

Historical numbers (2026-07-19, per-user SQLite, pre-port , do not treat as current)

Kept for reference only , this measured withWriteLock/getUserDb/per-user-SQLite-file architecture that this fork replaced.

Hardware

SpecValue
CPUAMD Ryzen 5 5600T, 6 cores / 12 threads @ 3500 MHz
RAM32GB (25GB available)
OSLinux 6.6.114 (WSL2 on Windows)
RuntimeBun v1.3.14
Workers12 (one per logical core via availableParallelism())

Configuration (pre-port)

  • reusePort: true on Bun.serve , kernel load-balances across 12 real OS worker processes (Bun.spawn, not threads)
  • ssr: 'data-only' on the _app layout route , server-side React rendering skipped entirely; thin shell + dehydrated JSON sent instead (still true in this fork , unaffected by the DB port)
  • Bootstrap TTLCache (10s, keyed on userId) , skips DB queries on cache hit (still true in this fork)
  • better-auth cookieCache (5 min) , session reads a signed cookie instead of hitting sessions/users tables (still true in this fork)
  • WAL mode + synchronous=NORMAL + mmap_size=256MB on all per-user SQLite DBs , gone in this fork, no per-user SQLite files exist; the equivalent tuning question is now Postgres connection-pool sizing (self-hosted) or Hyperdrive's own pooling behavior (Workers), neither benchmarked yet
  • Landing page (/) prerendered static HTML , zero SSR cost (still true in this fork)

Results (repeat=3, full mode, 1000-user pool, per-user SQLite architecture)

Main suite:

EndpointConcReq/sP50msP95msP99msSuccessAvgBodyCoV%
Login page101,2386.818.026.2100.0%8.5KB±4%
Login page501,80325.354.974.6100.0%8.5KB±6%
Login page1002,40937.380.2111.8100.0%8.5KB±3%
Landing page108,6110.64.36.6100.0%75.4KB±2%
Landing page5012,1813.49.212.2100.0%75.4KB±0%
Landing page10010,6708.817.221.7100.0%75.4KB±3%
Dashboard1080610.827.337.8100.0%6.8KB±2%
Dashboard501,78024.161.384.9100.0%6.8KB±3%
Dashboard1002,08143.791.2128.0100.0%6.8KB±2%
Billing1059315.830.740.4100.0%7.3KB±2%
Billing501,08842.090.1116.7100.0%7.3KB±1%
Billing1001,22974.6157.8205.7100.0%7.4KB±1%
Settings101,1008.019.024.6100.0%7.4KB±5%
Settings501,61428.561.287.7100.0%7.4KB±3%
Settings1001,88648.1103.7142.4100.0%7.4KB±2%

Admin Analytics (single authenticated admin session, shared-DB aggregation cost over subscriptions/payments/mrr_snapshots , this part of the query cost model carries over to this fork unchanged, since those tables were always in the shared DB, both before and after the port):

EndpointConcReq/sP50msP95msP99msSuccessAvgBodyCoV%
Admin Analytics1072712.725.632.8100.0%8.5KB±4%
Admin Analytics501,42331.770.493.8100.0%8.5KB±2%
Admin Analytics1001,64554.8124.1166.1100.0%8.5KB±1%

Thresholds: P95 max 500ms, min success 95%, min throughput 50 req/s. All passed at the time, on the per-user-SQLite architecture this fork no longer uses.


What was wrong before today (pre-port)

Two independent, compounding bugs in scripts/seed-loadtest.ts / src/routes/api/loadtest/auth.ts / tests/load-test-admin.ts meant every number in the base template's benchmark history before 2026-07-19 was invalid. Kept here as a methodology postmortem , this class of bug (a broken auth-cookie signature silently redirecting to /login and getting counted as a passing 200) is DB-architecture-agnostic and worth knowing before trusting any future load test, on either template.

Bug 1: missing cookieCache cookie. The loadtest scripts signed only the session_token cookie, never better-auth's cookieCache cookie (session_data) that a real interactive login also sets. Without it, auth.api.getSession() fell through to real DB reads on every request, cache-hit or not. Fixed by signing a matching session_data cookie in the same HMAC/base64url format better-auth itself uses.

Bug 2: missing __Secure- cookie prefix (much larger impact). Under a production build, better-auth requires every session cookie to carry a __Secure- prefix whenever its baseURL resolves to an https:// URL. Every cookie the loadtest scripts hand-signed used the bare, unprefixed name, so auth.api.getSession() never found the cookie at all , _app.tsx's beforeLoad then correctly redirected to /login.

Why this went undetected for so long: @bitclaw/loadtest follows redirects and only checks the final HTTP status code. A rejected session silently 307-redirects to /login, the tool follows it, gets a 200, and counts it as a success , with a body indistinguishable from a real page unless you specifically compare body sizes across endpoints.

Fix: src/lib/secure-cookie-prefix.server.ts is the single source of truth for this logic, mirroring auth.ts's exact isDev/baseURL branching so it can't drift between auth.ts and the loadtest call sites again , this file is DB-architecture-agnostic and needed no change during the Postgres port.

Lesson for next time: the diagnostic signal was there the whole time in the AvgBody column , every authenticated endpoint reporting an identical body size to the unauthenticated Login page is a redirect, not a coincidence. Cross-check body sizes across endpoints, not just success percentages, before trusting a "100% success" load test result , on this fork's re-ported harness too, once it exists.