AI invoice extraction you can trust: check the numbers

Getting a model to read the fields off an invoice is no longer the hard part. Current models read PDFs and phone photos directly and return vendor, date, line items and totals in whatever structure you ask for. The hard part is getting a finance team to act on those numbers without checking every one by hand, because in accounts payable a wrong number does not stay on the screen. It becomes a payment.

Why 95% accuracy is not good enough

Extraction quality is usually quoted as one number: 95% accuracy, sometimes 98%. That figure is almost always per field. An invoice has a dozen fields that matter: vendor, invoice number, date, purchase order number, subtotal, tax, total and a few line items. If each field is right 95% of the time and the errors are independent, the chance that all twelve are right is 0.95 to the power of 12, about 54%. Almost every second invoice would have at least one wrong field.

And the cost of a wrong field is not average. A transposed digit in a total becomes a wrong payment, a wrong payment becomes a reconciliation problem, and enough of those become questions from an auditor. So the goal is not a better accuracy number. It is a way to know which invoices are right.

Reading, extracting, checking

It helps to think of three steps, each failing in its own way:

Reading the page: turning pixels into text. Close to solved for clean PDFs, weaker for faded thermal paper, stamps over the totals and photos taken at an angle.
Extracting the fields: deciding which number is the total and which is the tax. This is where language models are strong, because they read layout and context instead of relying on fixed templates.
Checking the result: making sure the extracted numbers agree with each other and with your own records. This is the step most projects skip, and it is the one that decides whether anyone can trust the output.

With a multimodal model the first two often happen in one call. The third is always yours to build.

Check the arithmetic the invoice already contains

An invoice carries its own proof. Quantity times unit price gives each line amount, the line amounts add up to the subtotal, and subtotal plus tax gives the total. None of this needs a model; it needs a few lines of code, and a document that fails any of these checks does not go through.

from decimal import Decimal

HALF_CENT = Decimal("0.005")

def problems(inv) -> list[str]:
    found = []
    for n, line in enumerate(inv.lines, start=1):
        # quantity × unit price, rounded, gives the printed amount
        if abs(line.qty * line.unit_price - line.amount) > HALF_CENT:
            found.append(
                f"line {n}: {line.qty} × {line.unit_price} ≠ {line.amount}"
            )
    if sum(line.amount for line in inv.lines) != inv.subtotal:
        found.append("line amounts do not add up to the subtotal")
    if inv.subtotal + inv.tax != inv.total:
        found.append("subtotal plus tax is not the total")
    return found

Three details matter here.

Use decimals, not floats. In floating point, 0.1 + 0.2 is not 0.3, and money checks built on floats fail in strange ways. Parse amounts from text straight into a decimal type.
Allow rounding only where it happens. A line like 1.5 hours at 33.33 is 49.995, printed as 50.00, so each line gets half a cent of slack. The sums are compared exactly, because most invoices print a subtotal that is exactly the sum of the printed lines. A vendor that rounds differently gets its own tolerance in configuration, the same way it gets its own tax rules. A general tolerance of a few cents on the totals would quietly let a misread digit through.
Say what failed. The function returns the broken rule, not just false, so the person who reviews the invoice sees "line 2: 3 × 19.99 ≠ 59.79" instead of hunting for the problem.

The arithmetic also catches mistakes the model is sure about. A model can misread a line three rows up and still report the subtotal with complete confidence, because it has no idea it misread anything. The numbers on the page are an independent check, and independence is what makes them worth having.

Check against your own records

An invoice can add up perfectly and still be wrong. The second set of checks compares it with what you already know:

Known vendor. The vendor exists in your records, and the tax ID matches.
Same bank details. If the bank account differs from the one on file, the invoice always goes to a person. "We have changed banks, please pay into this account" is one of the most common invoice frauds, and no amount of extraction accuracy protects against it.
Not a duplicate. The same vendor with the same invoice number, or the same amount and date, has not been processed already.
Matches the order. If you use purchase orders, the invoice matches the order and, where you track deliveries, what was actually received. Accounts payable teams call this two-way or three-way matching.

Route by what failed

With both sets of checks, every invoice lands in one of three places:

everything checks out: it goes straight through;
the numbers agree but something is uncertain, such as a field the extractor could barely read or a vendor seen for the first time: a person confirms instead of retyping;
a check fails: it becomes an exception, with the failed rule attached.

If your extraction returns a confidence score per field, use it for the middle group. Dedicated document services return such scores; a language model's own estimate of its confidence is a much weaker signal, which is one more reason to lean on the arithmetic.

Real invoices are messier than the demo

Scans arrive skewed, stapled and stamped. Some invoices show a total in euros and a converted amount in dollars, and a check that does not know which field is in which currency will flag a valid invoice. Some vendors include tax in each line, some add it once at the bottom, and some jurisdictions have two tax lines that are summed separately. New templates keep arriving.

The way through is to make assumptions explicit. Currency is a field you extract and carry through every calculation, never a default. Tax handling is configured per vendor once you have seen a few of their invoices, because vendors are consistent with themselves even when they differ from everyone else. And when a document is genuinely unreadable, it goes to a person at once, instead of producing a confident guess. A system that cleanly refuses a small share of documents is worth far more than one that invents numbers for the same share.

A person in the loop should not be a permanent tax. Every exception a reviewer resolves teaches the system something: this vendor writes dates day first, this template puts freight below the tax line, this scanner drops the first digit on faded totals. Those become vendor rules and better prompts, and fewer documents need a person over time.

Trust is earned per vendor, not granted to everything at once. Let invoices from vendors whose documents have passed every check for a few months go through without review, keep watching the rest, and widen the trusted group as the evidence builds up.

Measure exceptions, not accuracy

The number that runs the operation is exceptions per hundred invoices: how many documents needed a person. It maps directly to hours of work, and a finance lead knows exactly what "six per hundred" means for their week. A healthy system shows that number falling over time, with the remaining exceptions being real problems a person should see: duplicates, invoices that do not match an order, changed bank details.

In short: let the model read the invoice, let code check the arithmetic with exact decimals, compare the invoice with your own records, send every failure to a person with the reason attached, and track how many invoices still need one.