/api/* routes co-deploy as Vercel serverless functions (or any Node host of your choice).
Stack at a glance
| Layer | Choice |
|---|---|
| Framework | Next.js 14 (App Router) + TypeScript 5.6 strict |
| Styling | Tailwind CSS 3.4, custom oklch palette, next/font/google (Bricolage Grotesque + Inter Tight) |
| ORM | Drizzle 0.36 + drizzle-kit migrations, pg.Pool singleton |
| Database | PostgreSQL (Supabase free tier in production) |
| Auth | NextAuth.js 4, Credentials provider, JWT session, bcryptjs cost 10 |
| LLM clients | Anthropic + OpenAI + Azure OpenAI adapters; Google stub |
| BYO-key store | Browser localStorage (probot.llm.key.v1, probot.llm.azure.v1); x-llm-api-key header per req |
| Markdown | react-markdown 9 + remark-gfm 4 + SafeLink for rel/target; no rehype-raw (XSS-safe) |
| Testing | Vitest 2.1 + @vitejs/plugin-react + Testing Library; node for .ts, jsdom for .tsx |
| Hosting | Vercel (primary); Render / Fly.io / Railway / AWS Lightsail / Docker for self-host |
High-level flow
Repo layout
Data model
Two tables. Both haveid uuid PRIMARY KEY DEFAULT gen_random_uuid(), created_at, updated_at.
users
| Column | Type | Notes |
|---|---|---|
username | varchar(30) | unique |
email | varchar(255) | unique |
hashed_password | varchar(255) | bcryptjs cost 10 |
llm_provider | varchar(20) | default "anthropic" |
llm_model | varchar(60) | nullable; provider-specific model id or Azure deployment |
email_verified | boolean | default false |
There is no
api_key column. The LLM API key lives only in browser
localStorage and rides the x-llm-api-key header per request. See BYO-key
flow.bots
| Column | Type | Notes |
|---|---|---|
user_id | uuid | FK → users.id ON DELETE CASCADE |
name | varchar(100) | |
headline | varchar(120) | nullable |
personality | varchar(20) | one of professional / creative / enthusiastic |
context_text | text | resume + bio; ≤ 50,000 chars |
suggested_questions | jsonb | string[], ≤ 6 entries, each ≤ 200 chars |
loading_messages | jsonb | string[]; default ["Thinking…", "Searching memory…", …] |
is_active | boolean | default true |
Provider abstraction
All providers implementLLMProvider.complete():
canary-key test enforces this at the route and provider layers.
Errors normalize into ProviderError with a category field:
invalid_key→ returns400 invalid_llm_keyto the callerrate_limit→ returns429 provider_rate_limitprovider_unavailable/timeout→ returns502 provider_unavailable
Request lifecycle (POST /api/chat/[botId])
- Content-Type check - must include
application/json. - Key transport -
readApiKey(headers)extractsx-llm-api-key; missing →400 missing_llm_key. - Body size cap - measured byte length ≤ 16,384.
- JSON parse + Zod validation (
message: string, 1..8000 chars). - Bot lookup -
bots.id+is_active = true. - Owner lookup - fetches
llm_provider+llm_model. - Rate limit - per-bot, 2-tier (short-window + long-window).
- Input sanitize - Unicode-normalize, then ~35 patterns (prompt-injection, role-override, credential-probe).
- Provider dispatch - Azure pulls extra
x-llm-azure-endpoint+ optionalx-llm-azure-api-versionheaders. - Provider call -
provider.complete({ system, userMessage, apiKey, model, extras }). - Output sanitize - 4 leakage checks.
- Respond -
{ reply }.
/api/chat/[botId] reference.