Matching and confidence

OCR is wrong in predictable ways. Every text lookup over a document is therefore a distance calculation rather than a comparison, and every extracted value carries a score.

You do not configure any of this from a rule — matching is fuzzy and case-insensitive by default, and the only knobs are Exact() and MatchCase(). This page explains what is happening underneath, so that when you read a confidence number you know what it is worth.

Why matching tolerates OCR error

OCR does not produce random errors. It produces the same confusions over and over, because the glyphs genuinely look alike. The matcher knows them and charges less for them:

Cost	Confused pair
0.3	l↔1, O↔0, o↔0, I↔1, I↔l
0.4	S↔5, s↔5, B↔8, $↔S, space↔_, ,↔.
0.5	G↔6, Z↔2, c↔e, n↔h, u↔v
0.6	%→9

Anything else — a missing character, an extra one, a genuinely different letter — costs the full 1.0.

The consequence is the whole design. SCHEDULE B-l sits 0.3 away from SCHEDULE B-1, while a real one-character difference sits 1.0 away. A tolerance tight enough to reject a wrong match is still loose enough to absorb the scan, which is why you can write the phrase as a human would say it and stop thinking about it.

What the confidence number means

Confidence is a weighted mean of heuristic factors, clamped to 0–1. Each factor is one signal about whether the read is trustworthy, and the weights are the system’s compiled-in opinion about which signals matter:

Signal	Weight	What it observes
OCR confidence	1.0	What the OCR engine itself reported
Validation	1.0	Whether the value passed its format check
Pattern match	0.9	How well it matched an expected shape
Alignment	0.8	How cleanly the value lines up with its label
Consistency	0.8	Whether it agrees with related values
Fuzzy match	0.7	How close the label match was
Proximity	0.6	How near the value sits to its label
Count	0.5	Found versus expected occurrences

Those weights are policy, not configuration. You cannot change them from a rule, and that is deliberate — a threshold you tune per document type is a threshold nobody can reason about.

A label-to-value read scores the label and the value separately and combines them, so a crisp value under a smudged label scores differently from the reverse. That is usually what you want: a value you cannot attribute confidently is not a value you should trust.

Reading it from a rule

bbox.Doc().Right("Loan Amount").Confidence()

And for a value that has been written to the output with its confidence attached:

when out.AttrNum("CDLoanAmount", "confidence", 1.0) < 0.7
then findings.Warn("check_amount", "extraction", "low confidence on loan amount");

See Writing output for the attribute surface.

Calibrate the threshold — do not guess it

Caution

A score of 0.8 does not mean “80% likely correct”. It is a weighted average of heuristics, and its relationship to actual correctness depends on your documents, your OCR engine and your schema.

Treat it as an ordering, not a probability. It reliably tells you that one value is more trustworthy than another; it does not tell you how often a 0.8 is wrong.

So do not adopt a round number as a review threshold. Pick one the way you would pick any operating point:

1. Extract a sample and record, for each value, its score and whether it was actually right.
2. Look at how correctness varies with score across that sample.
3. Choose the point where the error rate crosses what your process can absorb.
4. Re-check it when the document mix or the OCR engine changes.

That is more work than picking 0.8, and it is the difference between a review queue sized by evidence and one sized by a guess. The scores are comparable across runs, so the sample only has to be collected once per document type.

Special regions

Signature, handwriting and barcode areas are detected during analysis, before any rule sees the document, so they are already attached when a rule asks. Detection is inference over the OCR geometry — position, aspect ratio, word density, isolation, proximity to a label like Signature — and none of it looks at the page image.

That bounds what it can tell you. It finds the place a signature belongs and reports whether something is in it. It does not verify a signature, read handwriting, or decode a barcode.

Caution

Barcode and QR detection emits regions but decodes nothing. A detected barcode region means “a barcode-shaped thing is here”, not a value. Anything needing the encoded data has to decode it outside the document pipeline.

What matching covers

Matching operates on the text and layout the analysis produced. Two boundaries are worth knowing before you plan around them:

• Barcodes are located, not decoded. You get the region; reading the symbol is a step you add.
• Corrections are applied, not explained. A corrected token tells you what it became, not the reasoning that produced it — so treat a correction as a value, not as evidence.

Page numbers

When several files are loaded together, pages are appended in argument order and renumbered from 1. A reported page number is a position in that argument list, not the page number stored inside the file — so shell glob order decides it. Pass files in a deliberate order when page numbers matter.

Continue with

• The Document API — what a rule actually calls
• Extraction — anchors, regions and field reads
• Tables — detection, headers and master-detail
• OCR input — conversion and text cleanup