The AI reads the invoice fine. Plan for the one it doesn't.
It's Monday, a few hundred invoices are waiting, and the demo read every sample perfectly. The part worth designing is the page it reads slightly wrong, the one that looks complete and nobody questions.
AI document extraction works well on clean, familiar documents, so design for the ones it gets wrong. Keep the extracted value separate from a checked value, show reviewers the spot on the page each field came from, check totals and matches against your own records automatically, send anything unusual or high-risk to a person, and log every correction so you can see which senders and fields fail. Test on your own messiest files, not the vendor's samples.
Monday, 8:40 a.m., accounts payable
Dana gets in early on Mondays because that's when the invoices pile up. Before the new tool, that meant a morning of typing: vendor, invoice number, date, line items, tax, total, due date. Now she opens the queue and every invoice is already filled in. It's a good feeling. The tool read the weekend's batch while nobody was looking.
She approves the first dozen quickly. They look right because they are right. A regular supplier, the same layout as always, a clean PDF from their billing system.
Number thirteen looks right too. Every field is filled. The total matches the line items. She approves it.
What she can't see from the screen is that this supplier switched billing software last month. Their new invoices put the date in day-month order, and the tool read the 3rd of October as the 10th of March. The due date, calculated from it, is now months off. Nothing errors. Nothing turns red. The invoice just quietly sits in the wrong place in the payment run.
Dana's story is made up, a composite. But it's the shape of the problem that matters, and it's why extraction projects go well or badly.
What actually went wrong
Extraction errors come in two kinds, and teams tend to plan only for the first.
The ones that announce themselves. A blurry scan the tool can't read. A missing total. A file that isn't an invoice at all. These are annoying but safe, because they stop the line and a person looks.
The ones that look finished. A date read in the wrong order. A subtotal picked up as the total. A credit note read as an invoice, so you pay instead of being paid. The same invoice sent twice with a slightly different number. A purchase order number from the wrong box. These look like success, and that's what makes them expensive.
Modern AI extraction is genuinely good. On familiar, clean documents it's often very good. But "very good" across a few hundred documents a week still means some wrong ones, and the wrong ones you'll care about are the plausible ones.
So the design question isn't "how accurate is it?" It's "what catches the plausible mistake before it becomes a payment?"
Decision guide
Errors that announce themselves, and errors that don't
| Consideration | Obvious failure | Plausible failure |
|---|---|---|
| Dates | No date found on the page. | Day and month read in the wrong order. |
| Amounts | Total missing or unreadable. | Subtotal picked up as the total. |
| Document type | A file that isn't an invoice is rejected. | A credit note read as an invoice. |
| Duplicates | Exact duplicate file flagged. | Same invoice resent with a slightly different number. |
An extracted number isn't a fact yet. It's a claim about a document, and something has to check it.
Keep "read" and "checked" apart
The most useful decision in an extraction system is a simple one: the value the AI read and the value you've confirmed are two different things, and the software should store them that way.
We made a similar call on Sutton|Place, in a very different setting. A central decision there was to keep what a seller supplied separate from verification results, so a supplied fact didn't become confirmed just by appearing in the platform. An extracted invoice field deserves the same treatment. It's a claim about a document until something checks it.
What does the checking? Mostly things you already own, not the model's confidence score:
- Arithmetic. Lines add up to the subtotal, the subtotal plus tax equals the total.
- Your records. The vendor exists, the purchase order is open, the amount matches what was ordered and received.
- Common sense rules. The invoice date isn't in the future or months in the past. The due date matches that vendor's terms.
- Duplicates. Same vendor, same amount, similar number, close dates.
- Never-automatic changes. If a vendor's bank details differ from what's on file, a person confirms it through a known contact, every time.
A model's confidence score can help with routing, but it measures how sure the model is, not whether it's right. Your checks are the ones you can trust.
How the work moves
From page to posted entry
- Read
AI extracts fields and records where on the page each came from.
- Check
Arithmetic, your records, duplicates and date rules.
- Route
Passed checks go to batch approval; failures and risk rules go to a person.
- Confirm
A checked value is stored separately from the extracted one.
- Learn
Corrections and reasons show which senders and fields need a rule.
Make review fast, not ceremonial
Review is where a lot of these projects quietly fail. Either everything goes to a person, and the tool saved nobody any time, or nothing does, and review is a rubber stamp.
The fix is to review the right things, and make each review take seconds:
- Send to a person only what failed a check or matches a risk rule, such as a new vendor, a high amount or changed bank details.
- Show the spot on the page each field came from, highlighted next to the value, so checking a date means a glance, not a hunt.
- Ask about the field, not the document. If only the due date is in doubt, ask about the due date.
- Make corrections quick, and ask why with a short pick list, such as "date format," "wrong total," or "not an invoice."
That last part matters most. Those corrections tell you exactly which vendors and fields cause trouble. After a few weeks, you'll know that one supplier always confuses the date, and you can fix it with a rule.
Test on your worst drawer
Vendor demos use clean samples. Your inbox doesn't. Before you choose a tool, or decide whether to build, gather a test set from your own files and make it ugly on purpose:
- Phone photos of paper invoices, taken at an angle.
- Faxes and scans of scans.
- Multi-page invoices, and multi-invoice PDFs.
- Credit notes, statements and reminders mixed in with invoices.
- Your biggest vendors and your weirdest ones.
- Anything in another language or currency you actually receive.
Write down the correct values for each, then run every candidate against the set. Count the plausible mistakes separately from the obvious ones. Ask each vendor what their tool does when it isn't sure, and whether you can see where on the page each value came from.
Back to Monday
Same queue, built differently. Dana opens it and sees most of the weekend's invoices have already passed every check and are waiting for a quick batch approval. Three are flagged. One has a date that doesn't fit the vendor's terms, with the date on the page highlighted beside the value. One is a credit note. One has new bank details and a note to call the vendor.
She fixes the date and picks "date format" as the reason. By Thursday someone has added a rule for that supplier. She calls about the bank details. The credit note gets filed as a credit note.
That's what we mean by AI that brings the right context into the moment work needs to happen, on our AI applications page, and it's usually built alongside the approval and posting steps in business process automation. If you're weighing an extraction tool, bring us your worst drawer of documents, and we'll help you test it properly.
We build alongside the people doing the work.
Bring us the problem you’re thinking about.