golemkit

AI & billing

AI & credits

All model access goes through the Vercel AI Gateway: models are plain “provider/model” strings, so switching between Claude, GPT, and Gemini is a string change — one API key locally (AI_GATEWAY_API_KEY), and on Vercel not even that (OIDC authenticates automatically).

The available chat models live in src/modules/ai/models.ts. That file is the whole multi-provider abstraction — edit the array, and the chat UI’s model picker follows.

The billable endpoint recipe

src/app/api/chat/route.ts is the reference implementation every metered AI endpoint copies:

  1. Authenticate — getSession(), 401 without one.
  2. Rate limit — rateLimit(“chat:” + userId, ...), 429 over the limit. Postgres fixed-window, no extra infrastructure.
  3. Pre-check credits — getCreditBalance(), 402 when empty, before paying the model provider.
  4. Stream — streamChat() from @/modules/ai, returned via toUIMessageStreamResponse().
  5. Meter — in onFinish, convert usage.totalTokens to credits with creditsForTokens() and spend them with chargeCredits().

One guard rail completes the recipe: every endpoint caps its input size (MAX_TEXT_INPUT_CHARS / MAX_PROMPT_CHARS in src/modules/ai/limits.ts, responding 413 over the cap). Metering is post-hoc and spends at most the remaining balance, so without a cap a user with 1 credit could submit an enormous input and the overrun would be yours. The cap bounds the worst-case cost of a single call.

Credits

Balances live in user_credits; every change also lands in the append-only credit_transactions ledger, in the same database transaction. Pricing is one constant — CREDITS_PER_1K_TOKENS in src/modules/billing/usage.ts — and new users start with a signup bonus (SIGNUP_BONUS_CREDITS).

When a charge drops a balance below LOW_CREDIT_THRESHOLD, the user gets a one-time “top up” email — sent on crossing, so nobody is spammed. Admins can also grant or deduct credits by email from /dashboard/admin; adjustments go through the same ledger as everything else.

RAG — including “chat with your PDF”

src/modules/rag is a complete retrieval pipeline on pgvector: documents are embedded with embedMany, retrieved by cosine similarity (HNSW index), and answered with inline citations. Try it at /dashboard/rag; the routes (/api/rag/ingest, /api/rag/ask) follow the same auth + metering recipe as chat — ingestion included: embedding tokens are charged like any model call, so adding context isn’t free.

/api/rag/ingest-pdf adds file upload: the PDF is parsed in-request with unpdf (no blob storage required), split into embedding-sized chunks by chunkText(), and fed into the same pipeline. Swap in Vercel Blob or S3 there if you want to keep the original files.

The AI Lab — two more recipes

/dashboard/lab applies the billable recipe two more times, each showing a different pattern (src/modules/ai):

  • Structured extraction — generateObject with a zod schema turns free text into typed JSON (action items with owner/due).
  • Image generation — image models report no token usage, so metering is a flat price per image (IMAGE_CREDITS).

The sixth recipe — the tool-calling agent — outgrew a Lab card: it now has a registry, streaming tool calls, human approval, and saved conversations. See the Agents page in this docs sidebar.

Sell your AI as an API

src/modules/apikeys lets users mint API keys (sk_…, SHA-256 hashed, one-time reveal, revocable at /dashboard/api-keys). /api/v1/summarize is the public variant of the billable recipe: step 1 swaps the session for Authorization: Bearer sk_… via authenticateApiKey(), and the caller’s credits are spent exactly like in the dashboard. Copy its shape to expose any feature as a metered API.

Consumers get conventional status codes: 401 invalid or revoked key, 402 out of credits, 413 input over the size cap, 429 rate limited. Each user can hold up to 20 active keys (MAX_API_KEYS_PER_USER).

Background jobs — the cron recipe

/api/cron/weekly-digest shows the scheduled-work pattern: a cron entry in vercel.json, a route guarded by CRON_SECRET (Vercel sends it as a Bearer token automatically), and a module query (getWeeklyUsageDigest()) that aggregates each user’s 7-day usage and emails a summary. The same run does the housekeeping — clearing expired rate-limit counters. Copy this shape for any scheduled job.