← Back to work
AI AutomationLab build · Tessera (fictional SaaS)2026

Lab build: an AI support copilot retrofitted onto a live admin app in one PR

A portfolio lab build on Tessera (a fictional SaaS): an AI support copilot retrofitted onto an existing admin app in a single PR, human-approved before any mutation, with accuracy from a real, repeatable eval.

Lab build: an AI support copilot retrofitted onto a live admin app in one PR
Role
Solo build: AI retrofit onto an existing app
Timeline
One PR (feat/ai-copilot) on top of the pre-ai app
Stack
Next.js · TypeScript · Vercel AI SDK · libSQL / Turso · Zod
  • 30/30 (100%)Tool-selection accuracy (30-command eval)
  • 36.7% → fixedSame eval before the prompt fix

The problem

Most 'add AI to your product' work isn't greenfield. It's retrofitting an agent onto an app that already exists, without rewriting it and without letting the AI take unsafe actions on live data.

The approach

  1. Started from a plain CRUD support-admin dashboard tagged `pre-ai` (zero AI code or dependencies), so the whole retrofit reads as one diff: `git diff pre-ai..feat/ai-copilot`.
  2. Added a copilot that reuses the existing data layer and audit log; its mutations flow through the same validate → write → audit_log pattern the manual UI already used.
  3. Made read-only tools auto-execute, and every mutating tool (refunds, trial extensions, emails) human-in-the-loop: the agent pauses and streams an approval card, nothing runs until an operator approves, and each action is written to the audit trail as 'copilot (approved by operator)'.
  4. Added an eval (npm run eval) over ~30 commands (read-only, mutating, and should-refuse/clarify cases), scoring tool-selection accuracy and argument match, exiting non-zero below 80%.

The results

A surgical, reviewable retrofit: AI added to an existing product in a single PR, every mutation gated by human approval and recorded in the same audit log as manual actions. Not a rewrite, and not a black box.

Have a process like this eating your team's time?

Book a 15-minute fit call →
BOOK A FIT CALL