structured extraction

Fill {{SCENARIO}} and {{DOMAIN}}, copy, paste into your coding agent.

# Kickoff — Structured Extraction Pipeline

Paste everything below into your coding agent after filling {{SCENARIO}} and {{DOMAIN}}.

---

You are working in this repo: Next.js 16 App Router + TypeScript + Tailwind, Clerk auth
(wired), Drizzle + Neon Postgres (schema `perficient`, `runs` + `documents` tables exist),
Anthropic SDK with `ANTHROPIC_API_KEY` in env. Deployed on Vercel; keep it deployable.

Build a structured-extraction pipeline for this scenario: {{SCENARIO}} (domain: {{DOMAIN}}).

## The bar: extraction is only "AI doing real work" if it survives messy input and
## proves its accuracy. Schema + validation + eval, not a demo of JSON.parse.

1. **Synthetic messy inputs.** Script: one Claude call generates 12-15 realistic raw
   {{DOMAIN}} documents (emails / reports / invoices / intake forms — whatever fits) as
   plain text, DELIBERATELY messy: inconsistent formats, missing fields, typos, two docs
   with conflicting info, one irrelevant document. Store in `documents`.
2. **Target schema.** `lib/extract/schema.ts`: a zod schema for the structured record
   (6-10 fields: strings, enums, numbers, dates, at least one array). Every field
   nullable — "not present" must be expressible, or the model will invent values.
   Mirror as a JSON Schema for the API's structured output.
3. **Extractor.** `app/api/extract/route.ts` (model `claude-sonnet-4-6`):
   - Structured output constrained to the JSON Schema.
   - System prompt: extract ONLY what is stated; null for absent fields — never guess;
     for each extracted field include a short `evidence` quote from the source.
   - zod-validate the response; on failure retry ONCE feeding the zod error back; second
     failure → row flagged `needs_review`, never silently dropped.
   - Add a `confidence` enum (high|medium|low) per record, reasoning-first field order.
4. **Batch run + review UI** at `/app`: table of all documents → extracted records,
   evidence quotes on hover/expand, `needs_review` rows highlighted, per-field null counts
   at the top. A human-review lane for low-confidence rows is the production story told in
   UI form. Log each extraction to `runs` (archetype "extraction") with token usage.
5. **Eval with known truth.** Because inputs are generated, generate the GROUND TRUTH
   alongside them (same call, separate field). `scripts/eval.ts`: field-level precision —
   % of extracted values matching truth, % of absent fields correctly null (the
   hallucination rate for extraction), and the conflict doc surfaced as `needs_review`.
   Print the table; run it live.

## Constraints
- Milestone after each step. The evidence-quote requirement is non-negotiable — it's the
  citation story in extraction form.
- All model calls through `lib/ai.ts`. Streaming for the batch endpoint or chunk the batch
  to stay inside serverless timeouts.