Lab build: an invoice extractor that flags every low-confidence field for review

The problem

Extracting invoices with an LLM is easy to demo and hard to trust: a confident-but-wrong model will happily return a clean-looking total that doesn't actually add up.

The approach

1. Upload a PDF or image and extract to a Zod-validated schema via Claude Haiku 4.5 using tool use, so the API enforces the output shape instead of us parsing free text.
2. Gave every field a confidence badge that is the worst of two independent signals: the model's own high/medium/low, and deterministic code checks (line items must sum to the total; dates must parse).
3. Made the field table editable, re-running the deterministic checks live, then exporting the corrected header + line items to CSV.
4. Graded the pipeline with an eval (npm run eval) over 30 real receipts from the third-party ICDAR-2019 SROIE dataset: field-level exact-match accuracy after normalization, exiting non-zero below 75%.

The results

A review-first extractor where a confident-but-wrong model can't paint a field green if the numbers don't reconcile, and whose accuracy is measured against an independent, third-party benchmark rather than invoices we wrote ourselves.