AI Chat

Warpkit ships a full AI chat feature at /chat with shared-Postgres persistence (scoped by user_id), OpenRouter streaming (cloud), and optional Chrome built-in LLM (local, on-device).

Architecture

Concern	Detail
SSE endpoint	`src/routes/api/v1/ai-chat.ts`
DB layer	`src/features/ai-chat/server/ai-chat.server.ts`
Server functions	`src/features/ai-chat/server/ai-chat.{queries,mutations}.ts`
Components	`src/features/ai-chat/components/`
Routes	`_app.dashboard.chat.tsx`, `_app.dashboard.chat.index.tsx`, `_app.dashboard.chat.$id.tsx`
Config	`config.ai` in `config.ts`

Conversations and messages are stored in the shared conversations and chatMessages pgTables. user_id is denormalized onto chatMessages too, rather than derived only via a join through conversations , this keeps every access point an explicit, single-table filter on the hot path instead of relying on a join to enforce ownership. The SSE endpoint streams tokens via @tanstack/ai + @tanstack/ai-openrouter, and needed no port work itself , it never touches per-user tables during the stream.

Correctness note (ownership check): every one of the five conversation access points (getConversationMessagesFn, insertMessage, updateConversationTitle, deleteConversation, getConversationById) now checks eq(conversations.userId, userId) (or eq(chatMessages.userId, userId)) explicitly. The base template's per-user-SQLite version only checked existence, not ownership, at these points , structurally safe there because per-user file isolation made a missing ownership check harmless. On shared Postgres this fork's version fixes a real write-access IDOR, not just a theoretical read leak , cross-tenant regression tests cover all five.

Configuration

// config.ts
ai: {
  model: 'openrouter/auto',       // any OpenRouter model slug
  systemPrompt: 'You are a helpful AI assistant.',
  localModel: true,               // show Chrome built-in LLM toggle
}

Set OPENROUTER_API_KEY in .env for cloud mode. Get a key at openrouter.ai.

Cloud Mode (OpenRouter)

Default. Streams via SSE to POST /api/v1/ai-chat. Requires OPENROUTER_API_KEY. Rate-limited at 30 req/min per IP, auth-gated.

config.ai.model accepts any OpenRouter model slug:

openrouter/auto , auto-routes to the best available model
anthropic/claude-3.5-sonnet , Claude 3.5 Sonnet
google/gemini-2.0-flash-001 , Gemini 2.0 Flash
meta-llama/llama-3.3-70b-instruct , Llama 3.3 70B

Local Mode (Chrome Built-in LLM)

Uses Chrome's on-device Gemini Nano via window.LanguageModel. No API key, no server round-trip, no token cost. Runs entirely in the browser.

Enable in config:

ai: { localModel: true }

When enabled, the chat header shows a toggle so users can switch between cloud and local. The toggle only appears when the browser reports the model is available , on unsupported browsers it is hidden automatically.

Why Chrome extensions don't need flags

Chrome extensions run in a privileged context. The user explicitly installed the extension and granted it permissions. Extension manifests can request AI access directly , no flags, no Origin Trial.

Web pages run in the regular renderer context with no default trust. window.LanguageModel on a web page requires either:

Manual flags (dev/local only): Enable in chrome://flags:
- #optimization-guide-on-device-model → Enabled BypassPerfRequirement
- #prompt-api-for-gemini-nano → Enabled
Restart Chrome, then trigger model download:
```
// Run in DevTools console once to download model weights:
await window.LanguageModel.create({
  monitor(m) {
    m.addEventListener('downloadprogress', e => console.log(e.loaded, e.total));
  }
});
```
Check chrome://components/ → "Optimization Guide On Device Model" should show a version number.
Origin Trial (production): Register your domain at googlechrome.github.io/OriginTrials, get a token, embed it:
```

<meta http-equiv="origin-trial" content="YOUR_TOKEN_HERE">
```
Or as an HTTP response header:
```
Origin-Trial: YOUR_TOKEN_HERE
```
Chrome then unlocks window.LanguageModel for eligible users on your domain automatically , no flags required.

Gemini Nano self-identity limitation

Gemini Nano is a small model. When asked "are you running in the browser?", it may describe itself as server-hosted , this is incorrect. The model's training data includes descriptions of the Gemma family as server-deployed, and small models don't reliably override baked-in self-knowledge via system prompts.

The model IS running on-device. The response is a training artifact, not a runtime fact. This is a known limitation of small LLMs. Larger cloud models (via OpenRouter) follow system prompt instructions more faithfully.

Availability states

window.LanguageModel.availability() returns:

Status	Meaning
`'readily'`	Model downloaded and ready
`'available'`	Model ready (alias, same behavior)
`'downloadable'`	Registered but weights not yet downloaded , trigger with `LanguageModel.create()`
`'after-download'`	Same as downloadable
`'no'`	Not supported on this browser/OS

Warpkit shows the toggle for 'readily' and 'available'. For 'downloadable', the user must trigger the download manually (dev setup only , Origin Trial handles this transparently in production).

Message persistence

Both cloud and local messages are saved to the shared chatMessages table via saveMessageFn. The conversation list updates automatically via invalidateQueries. Messages load from the DB on route navigation via the route loader.

Auto-titling: the first user message (up to 60 chars) becomes the conversation title.

Markdown rendering

Assistant messages render GFM markdown via the existing unified + remark-gfm + rehype-sanitize pipeline (src/lib/markdown.ts → renderChatMarkdown). Streaming messages render as plain text to avoid broken partial-markdown flicker. Rendering is memoized per message so the pipeline only runs when message content changes, not on every keypress.