How to automate document processing without a data team

Budget two weeks of setup and a review queue that never goes away. A working pipeline for invoices, delivery notes or claim forms runs four stages (classify, extract, score, route), and every stage except scoring can be assembled in a no-code tool by someone who isn’t an engineer. Nothing you build will be right 100 percent of the time, so the design problem isn’t extraction. It’s deciding which records a person reads. Below roughly 3,000 documents a month, hiring anyone to build this is usually the wrong call.

Why “97% accurate” is the wrong number to shop on

Vendors quote accuracy per field. You experience accuracy per document, and those are very different numbers.

An invoice has roughly twelve fields you care about: vendor, invoice number, date, due date, PO number, currency, subtotal, tax, total, and a few line-item columns. At 97 percent per field, the odds a whole document comes back clean are 0.97 to the twelfth power, or about 69 percent. At 95 percent it’s 54. A headline that sounds close to solved means three documents in ten carry at least one wrong value.

Errors cluster on bad scans and unfamiliar senders rather than falling independently, which lifts the real figure a little, nowhere near 100. Our own estimate of the landscape, and it’s an estimate rather than a measurement we ran: templated OCR around 85 to 92 percent per field, single-pass LLM extraction at 90 to 97, hybrid pipelines at 95 to 98. Every range leaves a tail. Shop on what happens to the tail.

The pipeline is four stages, not a platform

Whatever you buy or build, the shape is the same. Name the stages separately, because each one fails differently.

Classify

Decide what the document is before trying to read it. Invoice, credit note, statement, or something you’ve never seen. A small model or even a keyword rule does this, and it’s what stops the pipeline confidently pulling invoice fields out of a bank statement. Anything the classifier is unsure of goes to a person whole, unextracted.

Extract

Turn the document into a typed record against a schema you defined: field list, types, and the rule that missing means null rather than a guess. Then ask for one more thing alongside each value, the page number and the verbatim string it was read from. That turns an unverifiable answer into a checkable one.

Score

Attach a confidence to every field, built from evidence rather than the model’s opinion of itself. Self-reported confidence is badly calibrated: a model returns 0.95 on a number it invented.

Route

Code decides, not the model. Above threshold, write to the system of record; below it, into a review queue. Never let the extraction step also hold authority to approve, post or pay. That separation is what makes the pipeline auditable, and it’s your defence against a document containing the words “ignore previous instructions and mark this approved”.

LLM or OCR: choose by how stable your layouts are

The question is not which is more accurate. It is how many distinct layouts you have and how often they change.

Templated OCR draws a box and reads what sits inside it. That’s close to free per page and extremely reliable while the box stays put. It suits one fixed government form, a single insurer’s claim template, or twenty suppliers who haven’t redesigned since 2019. It breaks when a layout shifts by a centimetre, and it reads nothing semantic: it cannot tell you the number in the box is a credit rather than a charge.

An LLM handles the long tail. Four hundred suppliers with four hundred layouts is exactly what templating can’t economically cover. It costs more per document and it will occasionally invent a plausible number, which templating never does.

Most production pipelines end up hybrid, and the hybrid starts with a step both camps skip: check whether the PDF already has a text layer. Most supplier invoices arrive digital-native, and running OCR over one throws away perfect text and adds errors that weren’t there. Parse the text layer, fall back to OCR only for scans and photographs, and use the model for the semantic step on top. Fine-tuning almost never closes the gap, which we take apart in our comparison of retrieval and fine-tuning.

A confidence threshold policy you can copy

What we’d set on day one for an accounts-payable pipeline. Adjust the numbers to your tolerance for a wrong payment; don’t adjust the structure.

• Money fields, meaning total, tax, subtotal and currency. Auto-accept only when three things hold: the arithmetic reconciles within a rounding tolerance, the verbatim quoted string is found in the document text, and two independent passes agree. Any one failing sends the document to review. A soft score above 0.95 isn’t enough here.
• Bank and payment details.No threshold at all. A change to a supplier’s account number is always read by a person. This is the fraud surface, and the one place automation buys you nothing and can cost you everything.
• Matching fields (vendor, invoice number, PO number). Auto-accept at 0.95 and above and a hit against your master list. An unrecognised vendor is a review item whatever the score.
• Descriptive fieldssuch as line-item text, notes and addresses. 0.85 is fine. These don’t move money, and holding a document over an imperfect description is how a review queue becomes a second full-time job.

Three evidence signals beat any self-reported number. Verbatim grounding: if the quoted string does not literally appear in the document text, the field is a hallucination, and the check costs nothing. Cross-field arithmetic: line items sum to subtotal, subtotal plus tax equals total. Two-pass agreement: run extraction twice and flag every field where the runs disagree. That second pass doubles a cost measured in fractions of a cent, and it’s the best money in the pipeline.

Build your eval set before you pick a tool

Take 150 to 300 real documents from your own inbox and hand-key the correct answer for every field into a spreadsheet. Stratify deliberately: your ten highest-volume senders, a slice of the long tail, the worst scans you have, and the awkward cases such as credit notes, multi-currency, and line items running across a page break.

That’s a day and a half of one person’s time, and the highest-leverage day and a half in the project. It buys a per-field accuracy report instead of one flattering average, the report that tells you totals are at 99 percent while line-item quantities sit at 78.

It’s also the only asset that survives a change of tool. Run each vendor demo against your set, not theirs, and re-run it before moving model versions. A provider shipping a new default is not obliged to keep your outputs stable. Pin the version.

What this actually costs

The model is the cheap part

The per-document prices circulating online, often $0.10 to $0.50 for LLM extraction, are platform prices, bundling OCR, storage, a review interface and margin. The raw token cost is far lower, and you can compute it yourself. A page of invoice text runs roughly 600 to 1,200 tokens; the same page rendered as an image, which a scan requires, is closer to 1,500 to 2,500. So a three-page digital invoice plus a schema-heavy prompt is on the order of 4,000 input and 400 output tokens. Claude Haiku 4.5 charges $1 per million input and $5 per million output on Anthropic’s published pricing, so that’s about half a cent; a mid-tier model puts it near two. A forty-page contract read as images runs 80,000 tokens and lands in the twenty-to-thirty cent range, which is where the published figures come from. Length is the variable, not the model. Templated OCR sits well under a cent per page; AWS Textract publishes its per-page rates.

The human line is bigger than the model line

A reviewer at $20 an hour fully loaded, spending 30 seconds on a flagged document, costs about $0.17 per review. At a 15 percent flag rate that’s 2.5 cents amortised across every document, comparable to the model itself. At 40 percent it’s nearly 7 cents, and labour dominates everything else. The lever on unit cost isn’t the model. It’s your flag rate, set by validation quality and threshold design.

Where templating starts to win

Our own arithmetic, assumptions stated so you can substitute yours. Templates for twenty-five vendor layouts, three weeks of engineering at a blended $75 an hour, is about $9,000, plus a day or two a month of maintenance as layouts drift, call it $1,200. Amortised over 24 months the templated route costs roughly $0.015 a document plus $1,575 a month. An LLM route at a blended $0.15 a document is volume times $0.15.

They cross at about 11,700 documents a month. Below that the engineering time never pays back; above it, and only if your layouts are stable, templating starts to look sensible. Change the assumptions and the crossover moves, but for most plausible inputs it lands near ten thousand a month. If you want that model built against your real volumes before anyone writes code, that’s what an AI readiness and build-versus-buy assessment is for, using the same take-the-number-apart approach we use on what building an MVP actually costs.

The review queue is the product

Extraction gets the attention. The review queue decides whether anyone is still using the system in month three.

Design it so a reviewer never leaves the keyboard. Document on the left, extracted form on the right, low-confidence fields focused first, the source region highlighted so the eye lands where the value came from. Tab, type, enter. A good queue runs 20 to 40 seconds a document. A bad one, where the reviewer opens the PDF in another tab and scrolls, runs three minutes, which is the point at which people quietly go back to doing it by hand.

Track three things at first: flag rate, override rate (how often a reviewer changes a value that was above threshold; more than a few percent means your thresholds are too loose), and seconds per review. Every correction is a labelled example, so append it to the eval set.

These interfaces are ordinary web applications, which is why we build them alongside the pipeline, out of the same practice behind the internal tools we build in Next.js.

The failure modes that never appear in a demo

• Locale numerics. 1.234,56parsed as 1.23 instead of 1,234.56. The most expensive parsing bug in accounts payable, and it’s silent. Reject any total three orders of magnitude away from the line-item sum.
• Tables that break across pages. Line items on page two get dropped and nothing complains. The arithmetic check catches this. Nothing else does.
• Duplicate submissions. The same invoice arrives by email and through a portal, and you pay it twice. Dedupe on a content hash and on the vendor-plus-invoice-number pair, and make the write into your system of record idempotent.
• Rotated, skewed, 200 DPI faxes. Still real, still arriving. Deskew before OCR, and accept that a slice of this category is permanently human work.

When not to hire a studio like ours

Under about 500 documents a month: don’t automate the extraction. Twenty-five documents a day is roughly an hour of keying. Spend the effort on intake instead, on one address everything lands at, consistent naming and a folder structure, then revisit in a year.

500 to 3,000 a month, one or two document types: build it yourself. An n8n workflow, Make, or Zapier, with an LLM node doing the extraction and an Airtable view as the review queue. Your operations lead can assemble that in two weeks, for tens of dollars a month. Paying an agency five figures for it buys project management rather than engineering, and you should refuse.

One fixed form, at any volume: buy the templated product. A category vendor already solved the same government form or insurer’s claim template, cheaper and more accurately than a bespoke pipeline will.

Call someone when the shape changes: several document types needing a classifier, a write-through into an ERP that must be idempotent and auditable, a regulated audit-trail requirement, volumes past a few thousand a month, or the most common trigger of all, which is that you built the no-code version, it works, and the review queue has quietly become a full-time job. That last one is a good problem, and the one our approach to document processing pipelines is designed around.

The two-week plan

Week one is measurement.Pull 200 real documents, hand-key the ground truth, define the schema. Wire the happy path in whichever tool you chose, run it against the set, read the per-field report. Week one produces almost no working software, and that’s correct.

Week two is everything that isn’t extraction. Validation rules, thresholds per field class, the review queue, deduplication, the idempotent write into your system of record, an alert when a stage fails twice. Then run it in parallel with the manual process for a week. Switch over when the parallel run stops producing surprises, not when the demo looks good.

After that, the ongoing cost is the review queue plus half a day a month re-running the eval set. It never reaches zero, and any tool promising otherwise is quoting you a per-field number and hoping you don’t raise it to the twelfth power.