← Back to work
Lab build: an AI support copilot retrofitted onto a live admin app in one PR
A portfolio lab build on Tessera (a fictional SaaS): an AI support copilot retrofitted onto an existing admin app in a single PR, human-approved before any mutation, with accuracy from a real, repeatable eval.

- Role
- Solo build: AI retrofit onto an existing app
- Timeline
- One PR (feat/ai-copilot) on top of the pre-ai app
- Stack
- Next.js · TypeScript · Vercel AI SDK · libSQL / Turso · Zod
- 30/30 (100%)Tool-selection accuracy (30-command eval)
- 36.7% → fixedSame eval before the prompt fix
The problem
Most 'add AI to your product' work isn't greenfield. It's retrofitting an agent onto an app that already exists, without rewriting it and without letting the AI take unsafe actions on live data.
The approach
- Started from a plain CRUD support-admin dashboard tagged `pre-ai` (zero AI code or dependencies), so the whole retrofit reads as one diff: `git diff pre-ai..feat/ai-copilot`.
- Added a copilot that reuses the existing data layer and audit log; its mutations flow through the same validate → write → audit_log pattern the manual UI already used.
- Made read-only tools auto-execute, and every mutating tool (refunds, trial extensions, emails) human-in-the-loop: the agent pauses and streams an approval card, nothing runs until an operator approves, and each action is written to the audit trail as 'copilot (approved by operator)'.
- Added an eval (npm run eval) over ~30 commands (read-only, mutating, and should-refuse/clarify cases), scoring tool-selection accuracy and argument match, exiting non-zero below 80%.
The results
A surgical, reviewable retrofit: AI added to an existing product in a single PR, every mutation gated by human approval and recorded in the same audit log as manual actions. Not a rewrite, and not a black box.
Have a process like this eating your team's time?
Book a 15-minute fit call →