E2E Testing

E2E Testing

Playwright-based browser tests for local development and a weekend GitHub Actions run (see CI). They validate full user flows that unit tests can't cover.

Prerequisites

  • .env.e2e configured (see setup below)
  • Chromium installed: bunx playwright install chromium

Setup

.env.e2e is committed with default test credentials matching the load test defaults. No setup needed unless you changed LOADTEST_EMAIL / LOADTEST_OTP in your .env.

The auth tests use /api/loadtest/auth to create a session without the OTP email flow. That endpoint must be enabled in .env:

LOADTEST_AUTH_ENABLED=true
LOADTEST_EMAIL=loadtest@example.com
LOADTEST_OTP=loadtest123

Running tests

make e2e          # migrate + seed the e2e DB, start its own server on :3100, run headless Chromium
make e2e.all      # same, all browsers
make e2e.ui       # headed Chromium (watch tests run)
make e2e.webkit   # headed WebKit/Safari
make e2e.report   # open last HTML report

e2e runs on its own port (PLAYWRIGHT_BASE_URL in .env.e2e, :3100), so it works while make dev is running on :3000. e2e/global-setup.ts fails loudly if something else already holds :3100. The spawned server's output goes to e2e/.tmp/server.log.

Don't poll a Bun dev server while it boots

Under Bun (bun --bun vite dev), any connection to the port while Vite is still initializing deadlocks it for good: the log stops after content-collections' "Start watching" and the server never listens. Node-run Vite doesn't do this, and connecting after the "Local:" line is fine. So global-setup waits for "Local:" in the server log before its first health probe. The same applies by hand: don't curl-loop a dev server right after starting it, and close browser tabs pointed at the port before restarting.

make e2e.ui and make e2e.webkit open a real browser window so you can watch tests execute. Useful for debugging failures visually.

Test structure

e2e/
├── playwright.config.ts    -- at project root
├── helpers/
│   ├── auth.ts             -- loginAsTestUser()
│   └── navigation.ts       -- gotoHydrated(), waitForHydration()
├── fixtures/
│   └── users.ts            -- TEST_USER from env vars
└── tests/
    ├── smoke.spec.ts       -- no auth required (landing, redirects, login page)
    └── auth/
        └── login.spec.ts   -- authenticated flows (skipped without .env.e2e)

Auth in tests

loginAsTestUser(page) from e2e/helpers/auth.ts:

  1. POSTs to /api/loadtest/auth with credentials from env
  2. Extracts the Set-Cookie header and applies it to the page context
  3. Navigates to /dashboard to warm up the per-user SQLite DB (first access runs migrations, 1-2s)

Auth tests are skipped automatically when LOADTEST_AUTH_ENABLED is not set, so make e2e still works without credentials (smoke tests run, auth tests skip).

Feature flags in e2e

.env.e2e sets E2E_ALL_FLAGS=true, and scripts/seed.ts then seeds every feature flag on except cookie_consent_enabled (its banner covers the page). Specs open gated pages directly instead of flipping flags through the admin UI first, which was slow and let specs rot unnoticed while their flag was off. Config-dependent features (credits, uploads, AI chat) render their "not configured" state, which the specs assert. Dev and prod seeds are unchanged.

The default-off path has one spec: e2e/tests/dashboard/flag-gate.spec.ts turns ai_chat_enabled off with setFeatureFlag and asserts the redirect. Only turn off a flag no other spec needs, or it races them.

Waiting for hydration (flake rule)

SSR'd buttons and inputs are clickable before React hydrates. A click or fill() that lands first is silently dropped: no error, the next locator just times out on the wrong state. waitUntil: 'networkidle' is NOT a hydration signal; on a slow CI runner the network goes quiet while hydration is still running (2026-10-24, runmist: onboarding wizard stuck on step 1 for 60s, reproduced locally on the first try under 6x CDP CPU throttle).

__root.tsx sets <html data-hydrated="true"> in a root useEffect (root effects run after every child effect in the hydration commit). Use the helpers in e2e/helpers/navigation.ts:

import { gotoHydrated, waitForHydration } from '../../helpers/navigation';

await gotoHydrated(page, '/onboarding');   // goto + wait for data-hydrated
await page.reload();
await waitForHydration(page);              // after reload or a popup's goto

Rules:

  • Every full page load followed by interaction goes through gotoHydrated. Client-side navigations (clicks, router.navigate) don't re-hydrate and need no extra wait.
  • Multi-step wizards: assert the step's heading is visible and the button toBeEnabled() before each click. Never chain blind clicks.
  • Dropdown triggers in tables other specs write to (admin users list): retry the click until the menu is open (openRowMenu in e2e/tests/admin/users.spec.ts). A parallel insert re-renders the row and the first click is lost (2026-10-24, warpkit-postgres).
  • Assert URLs with await expect(page).toHaveURL(...), not expect(page.url()), so client-side redirects get retried.
  • Repro a suspected timing flake by throttling the CPU, not by looping on a fast box (runmist: 232/232 passed locally before the throttle repro):
const cdp = await page.context().newCDPSession(page);
await cdp.send('Emulation.setCPUThrottlingRate', { rate: 6 });

Workers

Playwright detects available CPU and memory and sets worker count automatically:

const localWorkers = Math.min(cpus, Math.max(1, Math.floor(memGb / 2)));

One worker per 2GB of RAM, capped at CPU count. No manual tuning needed.

WSL2 note

Playwright's health checks use Node.js HTTP which resolves localhost to ::1 (IPv6) on WSL2. Vite only binds 127.0.0.1. Config uses http://127.0.0.1:3100 explicitly to avoid connection timeouts.

Keeping specs current

e2e isn't in CI, so specs drift when the UI changes. Run make e2e after UI changes. Patterns the suite relies on:

  • Demo features seeded off (credits, uploads, AI chat) are switched on in the spec's beforeEach with setFeatureFlag() from e2e/helpers/flags.ts (through the real admin page) and back off in afterAll.
  • Config-gated UI (Stripe prices, S3) is asserted as "either the real control or the not-configured state", never one of them only.
  • Wait for networkidle before clicking anything that needs hydration.

CI

Every app built on the template ships .github/workflows/e2e.yml: it runs make e2e once a week (Saturdays 07:17 UTC) and on demand (Actions tab, or gh workflow run e2e.yml). No setup needed after pushing to GitHub. It's not on every push or nightly: the suite takes a few minutes, and the free plan caps private repos at 2,000 Actions minutes a month, so a weekly run costs well under 5% of that. ci.yml stays the fast gate. Weekly is enough to catch spec drift before it piles up: forks that ran e2e only locally drifted for weeks without anyone noticing.

  • Chromium headless shell only, cached by Playwright version. A cache hit skips the download and the apt deps (the runner image has them).
  • node_modules/.vite is cached, so the dev server doesn't pre-bundle every dependency on the first page load.
  • Caches are saved even when the job fails (actions/cache/restore + save with always()).
  • On failure the Playwright report, traces (recorded on the first retry) and e2e/.tmp/server.log are uploaded as the e2e-report artifact (7 days).
  • CI has no .env, so the workflow sets the throwaway BETTER_AUTH_* values local runs read from it.

The dev server is started with --host from PLAYWRIGHT_BASE_URL (127.0.0.1). Without it Vite listens on localhost, which is ::1 on GitHub's runners, and every probe to 127.0.0.1 is refused.