golemkit

AI & billing

AI & credits

All model access goes through the Vercel AI Gateway: models are plain "provider/model" strings, so switching between Claude, GPT, and Gemini is a string change — one API key locally (AI_GATEWAY_API_KEY), and on Vercel not even that (OIDC authenticates automatically).

The available chat models live in src/modules/ai/models.ts. That file is the whole multi-provider abstraction — edit the array, and the chat UI's model picker follows.

The billable endpoint recipe

src/app/api/chat/route.ts is the reference implementation every metered AI endpoint copies:

  1. Authenticate — getSession(), 401 without one.
  2. Rate limit — rateLimit("chat:" + userId, ...), 429 over the limit. Postgres fixed-window, no extra infrastructure.
  3. Pre-check credits — getCreditBalance(), 402 when empty, before paying the model provider.
  4. Stream — streamChat() from @/modules/ai, returned via toUIMessageStreamResponse().
  5. Meter — in onFinish, convert usage.totalTokens to credits with creditsForTokens() and spend them with chargeCredits().

Credits

Balances live in user_credits; every change also lands in the append-only credit_transactions ledger, in the same database transaction. Pricing is one constant — CREDITS_PER_1K_TOKENS in src/modules/billing/usage.ts — and new users start with a signup bonus (SIGNUP_BONUS_CREDITS).

When a charge drops a balance below LOW_CREDIT_THRESHOLD, the user gets a one-time "top up" email — sent on crossing, so nobody is spammed. Admins can also grant or deduct credits by email from /dashboard/admin; adjustments go through the same ledger as everything else.

RAG — including "chat with your PDF"

src/modules/rag is a complete retrieval pipeline on pgvector: documents are embedded with embedMany, retrieved by cosine similarity (HNSW index), and answered with inline citations. Try it at /dashboard/rag; the routes (/api/rag/ingest, /api/rag/ask) follow the same auth + metering recipe as chat.

/api/rag/ingest-pdf adds file upload: the PDF is parsed in-request with unpdf (no blob storage required), split into embedding-sized chunks by chunkText(), and fed into the same pipeline. Swap in Vercel Blob or S3 there if you want to keep the original files.

The AI Lab — three more recipes

/dashboard/lab applies the billable recipe three more times, each showing a different pattern (src/modules/ai):

  • Structured extraction — generateObject with a zod schema turns free text into typed JSON (action items with owner/due).
  • Image generation — image models report no token usage, so metering is a flat price per image (IMAGE_CREDITS).
  • Tool-calling agent — generateTextwith typed tools reading the user's real balance and ledger, bounded by stopWhen: stepCountIs(n). Swap the demo tools for your domain's queries.

Sell your AI as an API

src/modules/apikeys lets users mint API keys (sk_…, SHA-256 hashed, one-time reveal, revocable at /dashboard/api-keys). /api/v1/summarize is the public variant of the billable recipe: step 1 swaps the session for Authorization: Bearer sk_… via authenticateApiKey(), and the caller's credits are spent exactly like in the dashboard. Copy its shape to expose any feature as a metered API.

Background jobs — the cron recipe

/api/cron/weekly-digest shows the scheduled-work pattern: a cron entry in vercel.json, a route guarded by CRON_SECRET (Vercel sends it as a Bearer token automatically), and a module query (getWeeklyUsageDigest()) that aggregates each user's 7-day usage and emails a summary. Copy this shape for any scheduled job.