From hOCR to extracted data

This page runs a single scanned page all the way through: an hOCR file from a recogniser, converted to the document model, inspected, then queried by a rule set that writes two values out. Every command, file and output below is real. Follow it top to bottom and you get the stated result.

The main run involves five files, all of them shown in full, the failure and split sections later on add a few more, each shown where it is used:

File	Role
`invoice-p1.hocr`	The recogniser’s output. The only thing you supply
`invoice-p1.bbox`	The document model, produced by `kis bbox create`
`invoice-p1.json`	Page facts, paired with the `.bbox` by basename
`rulesdir/invoice.grl`	The rule set
`invoice-p1-hist-audit.json`	The history and audit sidecar, written when `--history` or `--audit` asks for it

What you need

Requirement	Why
The `kis` CLI	Every command here
The rules plugin, installed separately	`kis bbox` and `kis rules` are proxy commands
An OCR engine that emits hOCR	Producing the input; nothing in this block performs OCR
A reachable `rules.svc` and a tenant JWT	Only for the decision step over HTTP

The commands are not on a bare kis install

kis bbox and kis rules forward their arguments to a separately installed rules plugin binary. Without it each one names the command you typed and the package that carries it, then exits non-zero:

Error: 'bbox' is not installed.
Install with: kvm install kis-rules

kis searches its own directory, then ~/.kisai/bin, then $KIS_PLUGIN_PATH, then $PATH. See the CLI reference.

The input

invoice-p1.hocr. One page of a 300-DPI US Letter scan, 2550 × 3300 units, six words.

<html><body>
<div class="ocr_page" id="page_1" title='image "invoice-p1.png"; bbox 0 0 2550 3300; ppageno 0'>
 <p class="ocr_par" title="bbox 300 240 1180 320">
  <span class="ocr_line" title="bbox 300 240 1180 320">
   <span class="ocrx_word" title="bbox 300 240 1180 320; x_wconf 96">INVOICE</span>
  </span>
 </p>
 <p class="ocr_par" title="bbox 300 520 1660 680">
  <span class="ocr_line" title="bbox 300 520 1660 580">
   <span class="ocrx_word" title="bbox 300 520 620 580; x_wconf 95">Invoice</span>
   <span class="ocrx_word" title="bbox 640 520 960 580; x_wconf 94">Number</span>
   <span class="ocrx_word" title="bbox 1200 520 1660 580; x_wconf 93">INV-2026-0431</span>
  </span>
  <span class="ocr_line" title="bbox 300 620 1660 680">
   <span class="ocrx_word" title="bbox 300 620 520 680; x_wconf 95">Total</span>
   <span class="ocrx_word" title="bbox 1200 620 1610 680; x_wconf 91">$1,250.00</span>
  </span>
 </p>
</div>
</body></html>

Step 1: convert

kis bbox create -i invoice-p1.hocr -l info

2026-07-27T17:34:12+05:30 INF successfully converted hocr to bbox app=kis.ai input=invoice-p1.hocr output=invoice-p1.bbox words=6
kis bbox create: converted 1 file → invoice-p1.bbox

With no -o the output is the input path with its extension replaced, so this writes invoice-p1.bbox beside the input.

Step 2: what the model looks like

invoice-p1.bbox in full. This is the document model on disk: one page, six words with raw top-left-origin coordinates, three lines and two paragraphs.

{
  "pages": [
    {
      "number": 1,
      "width": 2550,
      "height": 3300,
      "words": [
        {
          "text": "INVOICE",
          "box": { "x0": 300, "y0": 240, "x1": 1180, "y1": 320 },
          "confidence": 0.96,
          "page": 1,
          "line_id": 1
        },
        {
          "text": "Invoice",
          "box": { "x0": 300, "y0": 520, "x1": 620, "y1": 580 },
          "confidence": 0.95,
          "page": 1,
          "line_id": 2
        },
        {
          "text": "Number",
          "box": { "x0": 640, "y0": 520, "x1": 960, "y1": 580 },
          "confidence": 0.94,
          "page": 1,
          "line_id": 2
        },
        {
          "text": "INV-2026-0431",
          "box": { "x0": 1200, "y0": 520, "x1": 1660, "y1": 580 },
          "confidence": 0.93,
          "page": 1,
          "line_id": 2
        },
        {
          "text": "Total",
          "box": { "x0": 300, "y0": 620, "x1": 520, "y1": 680 },
          "confidence": 0.95,
          "page": 1,
          "line_id": 3
        },
        {
          "text": "$1,250.00",
          "box": { "x0": 1200, "y0": 620, "x1": 1610, "y1": 680 },
          "confidence": 0.91,
          "page": 1,
          "line_id": 3
        }
      ],
      "lines": [
        {
          "id": 1,
          "text": "INVOICE",
          "box": { "x0": 300, "y0": 240, "x1": 1180, "y1": 320 },
          "confidence": 0.96,
          "word_start": 0,
          "word_end": 1
        },
        {
          "id": 2,
          "text": "Invoice Number INV-2026-0431",
          "box": { "x0": 300, "y0": 520, "x1": 1660, "y1": 580 },
          "confidence": 0.94,
          "word_start": 1,
          "word_end": 4
        },
        {
          "id": 3,
          "text": "Total $1,250.00",
          "box": { "x0": 300, "y0": 620, "x1": 1610, "y1": 680 },
          "confidence": 0.9299999999999999,
          "word_start": 4,
          "word_end": 6
        }
      ],
      "paragraphs": [
        {
          "id": 1,
          "text": "INVOICE",
          "box": { "x0": 300, "y0": 240, "x1": 1180, "y1": 320 },
          "confidence": 0.96,
          "line_start": 0,
          "line_end": 1
        },
        {
          "id": 2,
          "text": "Invoice Number INV-2026-0431\nTotal $1,250.00",
          "box": { "x0": 300, "y0": 520, "x1": 1660, "y1": 680 },
          "confidence": 0.9349999999999999,
          "line_start": 1,
          "line_end": 3
        }
      ]
    }
  ]
}

The file itself puts each box field on its own line; the boxes are folded onto one line here and nothing else is changed.

Reading it against the hOCR:

In the file	Came from	Note
`width` / `height` `2550` × `3300`	The `ocr_page` `bbox`, ignoring the `image "…"` property	`ppageno` is never read; `number` is the position in document order
`box` on each word	That word’s own `bbox`	Raw OCR units, top-left origin, no normalisation
`confidence` `0.96`	`x_wconf 96`	Divided by 100
`line_id`	A per-page counter over `ocr_line` elements, stamped onto their words	Dense and 1-based per page; unrelated to any id in the hOCR
`lines[].word_start` / `word_end`	Half-open range into `words`	Line 2 is words 1, 2 and 3
`lines[].text`	Member words joined with a space	Never re-derived later: see below
`paragraphs[].line_start` / `line_end`	Half-open range into `lines`	Paragraph 2 is lines 2 and 3
`paragraphs[].text`	Member lines joined with a newline
`confidence` `0.9299999999999999`	The arithmetic mean of `0.95` and `0.91`	Binary floating point; compare with a tolerance, never with `==`

The full type reference, every field, method and coordinate rule, is the document model and its API reference.

Step 3: inspect before writing any rules

kis bbox loads a .bbox file, analyses it, prints what it found and runs nothing. Use it to confirm the anchors you are about to write rules against actually exist.

kis bbox phrase -p "Invoice Number" invoice-p1.bbox

Found 1 match(es) for phrase "Invoice Number"

=== Match 1 (Page 1) ===
Location: (300.0, 520.0) - (960.0, 580.0)
Line: 2

    INVOICE
 >> Invoice Number INV-2026-0431
    Total $1,250.00

The match box (300,520)-(960,580) is the merged Invoice + Number block, exactly the anchor a rule will resolve. Matching is case-sensitive unless you pass --case-insensitive.

kis bbox snippets --snip "header|Invoice Number|Total" invoice-p1.bbox

{
  "header": "Invoice Number INV-2026-0431"
}

kis bbox tables invoice-p1.bbox

=== Table 1 (Page 1) ===
Dimensions: 3 rows x 2 columns
Headers: INVOICE |

INVOICE        |
---------------+--------------
Invoice Number | INV-2026-0431
Total          | $1,250.00

Two label/value rows in a consistent left-and-right layout are enough for table detection to fire, and it pulls the page banner in as a header row. That is to check before you reach for tables on a document that is not really tabular.

Step 4: pull the values out with a rule set

Two files, both named for the same prefix. kis rules --with-bbox pairs <prefix>.json (the facts) with <prefix>.bbox (the document); --require-bbox drops an input that has no .bbox.

invoice-p1.json:

{
  "source": "scan-2026-07-27",
  "supplier_id": "SUP-118"
}

rulesdir/invoice.grl:

rule ReadInvoiceHeader "Invoice number and total from the scanned page" salience 100 {
    when
        bbox.Has("page")
    then
        out.Set("invoice_number", bbox.Get("page").TextRightOf("Invoice Number"));
        out.Set("total", bbox.Get("page").TextRightOf("Total"));
        out.Set("pages", bbox.Get("page").PageCount());
        out.Set("lines", bbox.Get("page").LineCount());
        audit.Log("ReadInvoiceHeader", "read the invoice header from page");
}

rule FlagMissingInvoiceNumber "The invoice number anchor produced nothing" salience 90 {
    when
        bbox.Has("page") && bbox.Get("page").TextRightOf("Invoice Number") == ""
    then
        out.Set("status", "invoice_number_missing");
        audit.Log("FlagMissingInvoiceNumber", "no text to the right of the Invoice Number anchor");
}

rule MarkComplete "Both header values were read" salience 80 {
    when
        bbox.Has("page") && bbox.Get("page").TextRightOf("Invoice Number") != "" && bbox.Get("page").TextRightOf("Total") != ""
    then
        out.Set("status", "complete");
        audit.Log("MarkComplete", "invoice number and total both present");
}

Three things in that file are load-bearing:

Written this way	Because
`bbox.Has("page")` in every `when`	`bbox.Get` returns nil for an unknown name and a method call on nil aborts the whole run
`bbox.Get("page").TextRightOf(...)` chained in one expression	`bbox.Get` hands back the analysed document, so the call reads straight through it. Inside a `then` you may bind it first, `d = bbox.Get("page");`, and call methods on `d`; a `when` has to chain
Salience `100`, `90`, `80`: all distinct	Equal salience gives a different firing order run to run

Each rule fires at most once per execution, so none of them needs a Retract.

page is the document name because that is what kis rules injects under. Change it with --bbox-name; kis rules --mode doc uses doc and kis rules -b defaults to bbox.

Run it:

kis rules -r rulesdir/ --with-bbox --history --audit invoice-p1

--- History for /work/invoice-p1.json ---
  1. [ReadInvoiceHeader] invoice_number = "INV-2026-0431"
  2. [ReadInvoiceHeader] total = "$1,250.00"
  3. [ReadInvoiceHeader] pages = 1
  4. [ReadInvoiceHeader] lines = 3
  5. [MarkComplete] status = "complete"


--- Audit for /work/invoice-p1.json ---
  1. [ReadInvoiceHeader] executed
  2. [ReadInvoiceHeader] read the invoice header from page
  3. [MarkComplete] executed
  4. [MarkComplete] invoice number and total both present

Result for /work/invoice-p1.json:
{
  "invoice_number": "INV-2026-0431",
  "lines": 3,
  "pages": 1,
  "status": "complete",
  "total": "$1,250.00"
}
Processed 1/1 input(s) successfully

FlagMissingInvoiceNumber was evaluated and did not match, so it contributed nothing. The two executed entries are added by the engine for every rule that fires; the other two are the rules’ own audit.Log calls.

TextRightOf("Invoice Number") resolved the merged anchor block at (300,520)-(960,580) and returned the nearest block to its right, INV-2026-0431 at (1200,520)-(1660,580). The same lookup against Total returned $1,250.00. Anchor resolution, direction, distance limits and the fuzzy variants are covered in extraction and matching.

The sidecar

Because the run passed --history and --audit, it also wrote invoice-p1-hist-audit.json:

{
  "history": [
    { "rule_id": "ReadInvoiceHeader", "key": "invoice_number", "value": "INV-2026-0431" },
    { "rule_id": "ReadInvoiceHeader", "key": "total", "value": "$1,250.00" },
    { "rule_id": "ReadInvoiceHeader", "key": "pages", "value": 1 },
    { "rule_id": "ReadInvoiceHeader", "key": "lines", "value": 3 },
    { "rule_id": "MarkComplete", "key": "status", "value": "complete" }
  ],
  "audit": [
    { "rule_id": "ReadInvoiceHeader", "message": "executed" },
    { "rule_id": "ReadInvoiceHeader", "message": "read the invoice header from page" },
    { "rule_id": "MarkComplete", "message": "executed" },
    { "rule_id": "MarkComplete", "message": "invoice number and total both present" }
  ]
}

Each entry is folded onto one line above; the file itself indents every field. The sidecar is written only when --history, --audit or --history-audit asks for it, and each such run appends rather than replacing, so a second run over the same prefix leaves eighteen entries where the first left nine. Send them somewhere else with --history-audit ./audit/.

Step 5: decide on the extracted values over HTTP

rules.svc serves no documents today: a request has no document input, and a rule set that names bbox does not load there. Document rules run through kis rules, as in Step 4. What crosses to the service is what the extraction produced. Extraction and the decision it feeds are separable, and the decision needs five values rather than five thousand words: extract on the CLI, then post the values to a rule set that names no document.

Step 4 wrote invoice_number, total, pages, lines and status. The decision needs two of them, alongside the supplier the page already carried. acceptdir/invoice-accept.grl names no document, so it runs the same on both surfaces. Keep it in its own directory: -r loads every .grl it finds into one rule set, so leaving it beside invoice.grl would pull the document rules of Step 4 into this run:

rule RequireInvoiceNumber "the request carried no invoice number" salience 100 {
    when
        in.Has("invoice_number") == false
    then
        out.Set("status", "invoice_number_missing");
        audit.Log("RequireInvoiceNumber", "no invoice_number fact on the request");
}

rule AcceptInvoice "both header values arrived" salience 90 {
    when
        in.Has("invoice_number") && in.Has("total")
    then
        out.Set("status", "accepted");
        out.Set("invoice_number", invoice_number);
        audit.Log("AcceptInvoice", "invoice number and total both present");
}

{ "invoice_number": "INV-2026-0431", "total": "$1,250.00", "supplier_id": "SUP-118" }

{ "invoice_number": "INV-2026-0431", "status": "accepted" }

On the CLI it runs over a file holding the same keys, extracted.json, with no -b and no --with-bbox:

kis rules -r acceptdir/ -i . -p "extracted.json"

Result for /work/extracted.json:
{
  "invoice_number": "INV-2026-0431",
  "status": "accepted"
}
Processed 1/1 input(s) successfully

On rules.svc, put invoice-accept.grl in the product’s rules/invoice-accept/ folder and ask for the decision; input carries the same three keys:

curl -X POST https://rules.example.com/rules/execute -H 'Content-Type: application/json' -H 'Authorization: Bearer <tenant-jwt>' -H 'X-Customer: acme' -H 'X-Product: erp' -H 'X-Env: prod' -H 'X-Tenant: eu1' -d '{"scope":"invoice-accept","input":{"invoice_number":"INV-2026-0431","total":"$1,250.00","supplier_id":"SUP-118"}}'

{
  "scope": "invoice-accept",
  "stamp": "sha256:…",
  "principal": { "id": "u-…", "ptyp": "user" },
  "out": { "invoice_number": "INV-2026-0431", "status": "accepted" },
  "findings": [],
  "actions": []
}

CLI	`rules.svc`
`extracted.json` top-level keys	`input`’s top-level keys
`-r acceptdir/`	`scope`, a folder of the product’s `rules/`
`Result for …` block	`out` in the `200` response
`--history` / `--audit` output	`options.history` / `options.audit` on the request

in.Has guards the optional facts here; the in readers also take a default and resolve dotted paths, so in.Str("invoice_number", "") reads a missing fact as "" rather than aborting. See Reading facts. Headers, status codes, the error codes and how rule sets reach the service are on running the service.

Recording a validation result

The run above records its outcome twice: a machine-readable status on out, which is what the caller sees, and a human-readable line on audit, which is what an operator reads. That is what FlagMissingInvoiceNumber and MarkComplete do above.

When the conversion produces nothing useful

The hOCR parser accepts almost anything and reports success. Every failure below exits 0. These are the four you will actually meet; the first and third are reproduced in full underneath. Only an input with no hOCR structure at all draws a warning at the default log level.

Symptom in `.bbox`	Cause	What to check
One word holding the whole file, zero box, confidence `0`	The input is not hOCR	The `has no hOCR structure` warning; `words=1` in the info log; `orphan_text` warning
Several words, all with zero boxes and confidence `0`	HTML with no `ocr` classes	`orphan_text` warnings, `width` and `height` both `0`
`width` and `height` are `0`, words are fine	The `ocr_page` has no `bbox`, or a malformed one	`missing_bbox` warning on `ocr_page`
A cluster of words sharing one identical oversized box	Word elements have no `bbox`	`missing_bbox` warnings on `ocrx_word`; confidence is forced to `0`

Run the conversion with -v -l info to see any of it:

kis bbox create -i notreally.hocr -v -l info

A plain text file fed to the converter. notreally.hocr is two lines with no markup at all:

Invoice Number INV-2026-0431
Total $1,250.00

2026-07-27T17:34:42+05:30 WRN parsing completed with warnings app=kis.ai input=notreally.hocr warningCount=2
2026-07-27T17:34:42+05:30 WRN hocr parsing warning app=kis.ai context="text not in word element, creating fallback word" element=text_node text="Invoice Number INV-2026-0431\nTotal $1,250.00" type=orphan_text
2026-07-27T17:34:42+05:30 WRN hocr parsing warning app=kis.ai context="no bbox available, using zero box" element=word text="Invoice Number INV-2026-0431\nTotal $1,250.00" type=no_bbox_fallback
warning: notreally.hocr has no hOCR structure (no ocr_page/ocr_line/ocrx_word elements found); output will be fallback text in zero-sized boxes. Is this really an hOCR file?
2026-07-27T17:34:42+05:30 WRN input has no hOCR structure; output will contain fallback text with zero-sized boxes app=kis.ai input=notreally.hocr
2026-07-27T17:34:42+05:30 INF successfully converted hocr to bbox app=kis.ai input=notreally.hocr output=notreally.bbox words=1
kis bbox create: converted 1 file → notreally.bbox

The warning: line is printed at every log level; the rest needs -v -l info.

{
  "pages": [
    {
      "number": 1,
      "width": 0,
      "height": 0,
      "words": [
        {
          "text": "Invoice Number INV-2026-0431\nTotal $1,250.00",
          "box": { "x0": 0, "y0": 0, "x1": 0, "y1": 0 },
          "confidence": 0,
          "page": 1
        }
      ]
    }
  ]
}

A page whose ocr_page title carries no bbox, with the words themselves intact. nopage.hocr is the second line of the invoice with the page bbox deleted:

<html><body>
<div class="ocr_page" id="page_1" title='image "nopage.png"; ppageno 0'>
 <span class="ocr_line" title="bbox 300 620 1610 680">
  <span class="ocrx_word" title="bbox 300 620 520 680; x_wconf 95">Total</span>
  <span class="ocrx_word" title="bbox 1200 620 1610 680; x_wconf 91">$1,250.00</span>
 </span>
</div>
</body></html>

kis bbox create -i nopage.hocr -v -l info

2026-07-27T17:34:50+05:30 WRN parsing completed with warnings app=kis.ai input=nopage.hocr warningCount=1
2026-07-27T17:34:50+05:30 WRN hocr parsing warning app=kis.ai context="page element has no bbox" element=ocr_page text= type=missing_bbox
2026-07-27T17:34:50+05:30 INF successfully converted hocr to bbox app=kis.ai input=nopage.hocr output=nopage.bbox words=2
kis bbox create: converted 1 file → nopage.bbox

width and height come out 0. Anchor queries still work, because analysis derives the page size from the words’ own extent, but that under-estimates the physical page, so every percentage region is over-scaled, and region and layout detection substitutes a fixed 612 × 792 instead. Fix the recogniser output rather than compensating in rules.

Checks, in order

words=N in the conversion log. Zero or one on a real page means the parse failed open. Add -v -l info and read the warnings.
width and height on each page. Zero means the page declaration was missing or malformed.
kis bbox phrase -p "<your anchor>". If the anchor is not found, no rule using it can work. Try --case-insensitive and --fuzzy before changing the rule set.
The match box in that output. Word merging may have glued your anchor to a neighbour, or split it; the box tells you which block a rule will actually resolve.
kis rules --history --audit. History shows which rule wrote which value; audit shows which rules fired at all.

Failures that produce no output at all

What you see	Cause	Fix
`no inputs resolved (...)`	Under `--require-bbox`, a `.json` exists without its `.bbox`	Both files must share a basename; the `warning: skipped …` line above names what was dropped
`... the when raised an error. got non existent key bbox`	The run carried no document: no `-b` or `--with-bbox`, or `--with-bbox` found no `.bbox` beside that input	Add `--with-bbox` or `-b`, and check the `.bbox` is there. `--require-bbox` drops an input without one instead
`... got rule engine execute panic on rule <name> ! recovered : runtime error: invalid memory address or nil pointer dereference`	A method was called on `bbox.Get("<wrong name>")` in a `then`	Match the document name exactly, and guard with `bbox.Has`
`... error while evaluating rule <name> ! recovered : runtime error: invalid memory address or nil pointer dereference`	The same nil call, but in a `when`	Same fix; only the wording differs by clause
An empty result and no error at all	Every `when` was guarded with `bbox.Has` and the document name was wrong	The guard is false, so no rule fires. Check the name against `--bbox-name`
`plugin bbox prepare: loading document page from <prefix>.bbox: loading OCR JSON from <prefix>.bbox: invalid character ...`	The `.bbox` is not JSON. A failed or truncated conversion	Re-run the conversion and check the word count in its info log

Hold the nil-pointer rows and the empty-result row together: guarding with bbox.Has turns a crash into silence. A run that returns {} with no error is the expected shape of a wrong document name, not evidence that the document was empty.

One failed rule aborts the entire execution and discards everything already collected, so a single unguarded call destroys the output of every rule that fired before it. Troubleshooting walks the rest.

Continue with

OCR ingestion: every accepted hOCR class, the warning types, text cleanup profiles
The document model: coordinates, blocks, lines, reading order
Extraction: anchors, directional capture, forms and checkboxes
Running rules over documents: the full bbox surface a rule can call
Writing rules: salience, Retract, and what compiles
Command line: every flag on kis rules and kis bbox
Running the service: POST /rules/execute, tenancy headers, status codes
API reference: the typed surface behind both