Lab build: a public Telegram support bot that routes 30/30 and cites its source docs
A portfolio lab build on Tessera (a fictional SaaS): a public Telegram support agent that answers from the product's help docs with the source article cited, looks up real ticket status, and escalates billing disputes to a human. Hardened for public traffic (webhook secret, idempotent updates, per-chat rate limits, a daily LLM budget) and scoring 30/30 routing and 27/27 retrieval hit@1 at $0 on the free Gemini tier.
- Role
- Solo build (Claude Code)
- Stack
- Next.js · TypeScript · Telegram Bot API · Gemini (free-tier gen + embeddings) · libSQL / Turso
- 30/30 (100%)Routing accuracy (30-intent eval)
- 27/27 hit@1Retrieval accuracy (27-question eval)
The problem
Public support bots are easy to demo and hard to run: they hallucinate answers, can't actually check a ticket, never admit when a human is needed, and fall over the moment real traffic (or Telegram's retry storms) arrives.
The approach
- Answers from the product's help docs with RAG and cites the source article inline, so a customer can verify the answer instead of trusting it. Out-of-scope questions get an honest refusal, not a guess.
- Looks up real ticket status on request, and routes billing disputes or an explicit 'get me a human' to honest escalation: the transcript is forwarded and the customer is told a person will follow up, rather than the bot pretending to resolve it.
- Hardened for public traffic: webhook secret auth, update_id idempotency so Telegram's retry storms can't double-process a message, per-chat rate limits plus a global daily LLM-call budget, and a degraded no-DB mode that keeps answering docs questions when the database is down.
- Guarded by two repeatable evals: npm run eval:routing over docs/ticket/escalate/refuse intents, and npm run eval:retrieval over the golden question set.
The results
A public bot you can actually message: it answers from real docs with citations, checks real tickets, hands billing disputes to a human honestly, and stays up under retry storms and load. Measured, not asserted, and running at $0 on the free tier.
Have a process like this eating your team's time?
Book a 15-minute fit call →