OCR and text acquisition

The Rules block ingests OCR output. It does not perform OCR.

You run a recogniser yourself, out of band. If it emits hOCR, kis bbox create converts that to a .bbox word-box file; if it already emits word boxes, write the .bbox JSON directly and skip the conversion. Either way the .bbox file is what loads into the document model that extraction, tables and matching operate on.

There is no bundled OCR engine

Nothing in this block executes a recogniser. There is no subprocess launch, no cloud OCR call, no image or PDF handling, no model file, no language data, and no engine-selection setting of any kind. Documents reach the model as engine output files only.

What must be installed

Component	Required for	Supplied by
The kis CLI	Every command on this page	The platform install
The rules plugin	kis bbox, kis rules	A separate plugin install
An OCR engine that emits hOCR	Producing the input files	You: entirely your own installation

Nothing else. No tessdata directory, no language pack, no model download, no locale configuration. Text handling is character substitution only. No dictionary, no language model.

kis bbox is a proxy command

kis bbox and kis rules are not built into the kis binary, it discovers and executes a separately installed rules plugin binary, passing your flags through unparsed. If the plugin is missing you get the command you typed back, plus an install hint:

Error: 'bbox' is not installed.
Install with: kvm install kis-rules

The search order is the directory holding the kis binary, then ~/.kisai/bin, then every entry of $KIS_PLUGIN_PATH, then every entry of $PATH. A candidate must have an executable bit set. Installing the CLI is not sufficient; install the rules plugin too.

The acquisition path

your OCR engine  ──▶  page.hocr  ──▶  kis bbox create  ──▶  page.bbox (JSON word boxes)
                                                                │
                          page.json (facts) ───────────────┬────┘
                                                           └──▶  kis rules --with-bbox  [--mode doc]

kis rules --with-bbox pairs a .bbox file with a same-basename .json facts file and runs rules over the pair, once per pair, or once over all of them with --mode doc. The command is covered in the CLI reference; how the document reaches a rule is covered in Rules over documents.

Documents stay on the CLI

rules.svc serves no documents today: a request has no document input, and a rule set that names bbox does not load there. Run document rules through kis rules, then send the values they extract to POST /rules/execute when a decision belongs on the service. See Running rules.svc.

hOCR ingestion

hOCR is HTML. It is parsed with a real HTML parser, not regex, and the parser is deliberately permissive about which producer emitted the file.

Recognised classes

This is the complete accepted vocabulary.

Level	Classes
Page	ocr_page
Content area (recursed into)	ocr_carea, ocr_photo, ocr_graphic, ocr_separator, ocr_noise
Block	ocr_par, ocr_textfloat, ocr_caption, ocr_header, ocr_footer
Line	ocr_line, ocrx_line, ocr_textline, ocr_header_line
Word	ocrx_word, ocr_word
Character	ocrx_cinfo, ocr_cinfo

Any other class beginning with ocr produces an unrecognized_class warning in verbose mode; its text is recovered only by the orphan-text fallback.

A page the parser accepts

<div class="ocr_page" id="page_1" title='image "/tmp/page_1.png"; bbox 0 0 2550 3300; ppageno 0'>
 <p class="ocr_par" title="bbox 100 100 400 160">
  <span class="ocr_line" title="bbox 100 100 400 160">
   <span class="ocrx_word" title="bbox 100 100 400 160; x_wconf 95">HEADER</span>
  </span>
 </p>
</div>

The page comes out 2550 × 3300; the word lands at {100,100,400,160} with confidence 0.95.

Mixed structure survives

Producers mix structure levels on one page. The parser runs a paragraph pass, then a bare-line pass, then a bare-word pass, each skipping subtrees the earlier pass already consumed, then, only if the page still has zero words, an orphan-text pass. A page containing an ocr_par, a bare ocr_line and a bare ocrx_word side by side yields three words, one paragraph and two lines, with nothing double-counted.

Title properties

An hOCR title is a ;-separated property list. Splitting honours double-quoted values, and the property name must match as a whole token.

bbox 100 100 200 200                                   -> {100,100,200,200}
image "/tmp/page_1.png"; bbox 0 0 2550 3300; ppageno 0 -> {0,0,2550,3300}
image "/tmp/scan_bbox.png"; bbox 0 0 2550 3300         -> {0,0,2550,3300}
x_bboxes 10 5 10 10 20 5 30 10; bbox 100 100 200 200   -> {100,100,200,200}
image "a;bbox 9 9 9 9.png"; bbox 1 2 3 4               -> {1,2,3,4}
image "/tmp/page.png"; ppageno 0                       -> no bbox
bboxes 1 2 3 4                                         -> no bbox
bbox a b c d; ppageno 7                                -> no bbox

A malformed bbox yields nothing rather than scavenging digits from elsewhere in the title.

Confidence

Property	Read as	Result
x_wconf N	Integer 0–100	N / 100
x_conf N	Float; divided by 100 if > 1	N or N / 100
Neither present	—	1.0
Word synthesised from fallback text	—	0.0

Every word carries confidence in the 0–1 range whatever the producer.

Confidence parsing is not token-exact

Unlike bbox, confidence is found by plain substring search for x_wconf / x_conf anywhere in the title, so another property or a filename containing those substrings can supply the value. And an absent confidence is indistinguishable from a perfect one, both read 1.0.

Words with no box

When a word element has no bbox, the parser synthesises the word using the nearest enclosing element’s box, line, paragraph, content area, or page, and sets confidence to 0.0. If no ancestor has a box either, the word gets a zero-size box at the origin, a cluster of words sharing one identical oversized box, and (0,0,0,0) boxes, are both real possible output.

The word-box JSON format

kis bbox create writes a .bbox file: JSON, indented two spaces, holding the serialised document. This is also the format to emit directly if your engine does not produce hOCR.

{
  "pages": [
    {
      "number": 1,
      "width": 612,
      "height": 792,
      "words": [
        {
          "text": "Borrower",
          "box": {"x0": 72, "y0": 100, "x1": 150, "y1": 115},
          "confidence": 0.98,
          "page": 1
        }
      ]
    }
  ]
}

words[].line_id is optional. lines and paragraphs arrays are optional and hold index ranges into words and lines respectively, see the document model for what those ranges mean.

Format detection

The extension is a hint, not a contract. The loader dispatches like this:

Extension	Behaviour
.json	JSON loader
.hocr	hOCR parser
.html, .htm	Sniffed for ocr_page, ocrx_word or ocr_word; hOCR if found, else JSON
Anything else, including .bbox	JSON first, hOCR second

This is why kis bbox subcommands accept a .hocr file even though their -i help string says “path to bbox file”.

Conversion does not detect format

kis bbox create bypasses this dispatch entirely and runs the hOCR parser on every file it is given, whatever the extension. Handing it a .json file does not round-trip: it parses as HTML and draws the has no hOCR structure warning. Handing it a .bbox file with no -o is refused, because the output would overwrite the input.

Converting hOCR

kis bbox create -i scan-page1.hocr

kis bbox create: converted 1 file → scan-page1.bbox

With no -o the output path is the input path with its extension replaced by .bbox (page.hocr.html becomes page.bbox). The confirmation line is printed at every log level; the word count needs -l info.

Flag	Default	Purpose
-i, --input	—	hOCR file, or a directory of hOCR files; files and directories may also be given as arguments
-o, --output	Input location	Output file (one input) or directory (several inputs, or a path ending in /)
-w, --workers	4	Parallel workers when there is more than one input
-v, --verbose	false	Report content the parser could not place
--clean	false	Apply the default text-cleanup profile
--clean-form	false	Apply the form-aware cleanup profile
--format	auto	The input format: auto or hocr; the output is always .bbox
--incremental	false	Skip inputs whose .bbox is already present and newer

kis bbox create -i scan-page1.hocr -o out/page1.bbox
kis bbox create -i hocr/ -o bbox/ -w 8 --clean
kis bbox create hocr/*.hocr -o bbox/ --clean-form
kis bbox create -i hocr/ -v

A directory is scanned for *.hocr and *.hocr.html, not recursively; a file named explicitly is converted whatever its extension. Matching nothing at all is an error: the command prints kis bbox create: no input files found to process. with what it looked for, and exits non-zero. Missing output directories are created, and each .bbox is written whole or not at all.

With several inputs, a failing file does not stop the run. Every file is attempted, then the command reports <n> of <m> hocr files failed to convert (first: <path>: <error>) and exits non-zero.

A non-hOCR file “succeeds” into a near-empty document

The HTML parser essentially never errors on arbitrary text. Feeding it a plain text file, a PDF dump or the wrong HTML gives exit code 0 and a .bbox holding one synthetic page of orphan-text words with zero-size boxes. When the input has no hOCR structure at all, the command prints warning: <file> has no hOCR structure (…) at every log level. Wrong HTML that carries some ocr classes draws no such warning: check the words count in the success log, and the parse warnings with -v.

HTML entities are decoded once, by the HTML parser: a page that literally reads <tag> converts to that text.

Parse warnings

Warnings are the only visibility you have into content the parser could not place.

Type	Meaning
no_structure	No ocr_page found and fallback extraction found no words
missing_bbox	A page or word element has no bbox
no_bbox_fallback	No box available anywhere; a zero box was used
empty_paragraph	Paragraph has text but no lines or words were extracted
empty_line	Line has text but no words were extracted
empty_word	Word element has no text content
orphan_text	Text outside any word element; a fallback word was created
unrecognized_class	An OCR class outside the accepted vocabulary
missed_content	Post-hoc diff of source text against extracted word text

missed_content compares all source text against all extracted word text, lowercased and punctuation-stripped, ignoring single characters, and reports the first 10 missed words.

Each warning renders as [type] element: "text" (context: ...), with text trimmed and truncated to 50 characters. Warnings carrying no text drop the : "text" part.

Warnings are discarded unless verbose is on

The accumulator checks the verbose flag before appending, so the warning list is always empty without -v. kis bbox create -v is the only place warnings surface. No other command prints them. Missing content is invisible by default.

Text cleanup and normalisation

Cleanup is pure text in, text out. It is character-level only. No dictionary, no language model, no language parameter.

Option	Effect
NormalizeUnicode	Unicode NFC
FixQuotes	23 quote, prime and guillemet variants, including backtick and acute, to ASCII ' or "
FixDashes	11 dash variants to -
FixLigatures	fi/fl/ff/ffi/ffl/st, Æ/Œ/IJ/ꜳ/ꝏ, and symbol expansions such as No., (P), SM, TM, (R), (C)
RemoveGarbage	Deletes 56 glyphs: bullets, arrows, box drawing, geometric shapes, stray diacritics, pilcrow, section sign
NormalizeFormIndicators	Checkbox and radio glyphs to [ ], [x], ( ), (x)
FixMisreads	Fullwidth ASCII to ASCII, Greek and Cyrillic homoglyphs to Latin, Roman numerals expanded, common Unicode space variants to a space, zero-width characters deleted
RemoveStrayPunct	16 X patterns to a single space
CleanWhitespace	Whitespace collapse
RemoveControlChars	Control characters removed
AggressiveClean	Everything outside printable ASCII becomes a space; newlines and tabs survive

The two profiles compose these options, applied in the order shown. RemoveGarbage reads PreserveFormIndicators to decide whether checkbox and radio glyphs count as garbage.

Profiles

Option	Default profile	Form-aware profile
NormalizeUnicode, FixQuotes, FixDashes, FixLigatures	on	on
RemoveGarbage, FixMisreads, CleanWhitespace, RemoveControlChars	on	on
PreserveFormIndicators	on	on
NormalizeFormIndicators	off	on
RemoveStrayPunct	off	off
AggressiveClean	off	off

--clean selects the default profile; --clean-form selects the form-aware profile.

--clean-form overrides --clean, and neither is on by default

The flags are an if/else-if: if --clean-form is set, --clean is ignored entirely. Neither is set by default, so conversion performs no text cleanup unless you ask for it.

Default cleanup is lossy where it matters most

FixDashes collapses en dash, em dash and the minus sign to a plain hyphen, so a leading minus and a dash become indistinguishable. FixQuotes maps the prime character to an apostrophe, losing feet and inches marks. FixMisreads maps the multiplication sign to x and the division sign and fraction slashes to /. RemoveGarbage strips the section sign, which is meaningful in legal text. All of these are on in the default profile.

Cleanup runs per word

Cleanup is applied to each word’s text, then each line’s text, then each paragraph’s text. Three consequences:

• Boxes are never adjusted, a word whose text is emptied by garbage-stripping stays in the output as an empty-text box.
• Word text and line text are cleaned independently. A line’s text was joined from the raw words at parse time, so after cleanup a line’s text is not necessarily its cleaned words rejoined.
• RemoveStrayPunct matches only a punctuation character with a space on both sides, so it is a no-op on word text, a single word has no spaces, but it does fire on line and paragraph text, where words were joined with spaces.

PreserveLineBreaks has no effect

Both profiles turn it on, but nothing reads it. CleanWhitespace collapses runs of three or more newlines to two regardless.

Cleanup is a conversion-time choice

--clean and --clean-form run inside kis bbox create, so what lands in the .bbox file is already cleaned. kis rules does not clean text: it takes the word text the .bbox file holds.

Engine output shapes

No surface selects an OCR engine: there is no CLI flag, no file format and no request field that names one. Convert your recogniser’s output to hOCR or to word-box JSON yourself, then supply it the way every other document is supplied, a path to kis rules. These three common shapes map onto word-box JSON, and the notes say what your conversion must handle.

Source	Shape it emits	What your conversion must do
PaddleOCR	Four-corner polygons, per-page result arrays	Reduce each polygon to an axis-aligned box; drop items with fewer than 4 corners
Tesseract TSV	Row records with level, position, size, conf	Keep only level-5 (word) rows with non-empty text; divide conf by 100
Docling	Word records with bbox [x0,y0,x1,y1] and score	Require at least 4 bbox values; score becomes confidence

PaddleOCR and Tesseract report page width and height. Docling does not, carry the page size from your own pipeline, or the page arrives sized 0 × 0.

Page dimensions

Page width and height are read from the ocr_page bbox, or from width and height in word-box JSON. If the hOCR has no ocr_page bbox, its page title is malformed, or the JSON page omits the dimensions, both are 0.

Page analysis seeds the statistics from the page declaration, and falls back to the bounding extent of the page’s words when the declaration is zero, flagging that the fallback was used. Percentage-based spatial queries scale by the statistics, not by the raw page record, so they still resolve, the fallback under-estimates the physical page, margins beyond the last word are lost, so percentage regions come out over-scaled. It is a bounded approximation rather than a dead query.

A page with no declared size silently becomes US Letter for layout

Region and layout detection does not use that fallback. It substitutes its own configured default of 612 × 792 points whenever a page reports zero width or height, so header, footer and margin bands are computed against a page that may be nothing like the real one, with no error anywhere. Header is the top 12%, footer the bottom 10%, margin up to 15% of width, all of a page size that was never measured.

Concurrency

Analysis runs in full for every execution that carries a document, and caches nothing between executions. Every detector, table, field, checkbox, region, runs on every page, and none of them can be turned off.

How many documents are in flight at once is set on the command line:

Where	Flag	Default	Effect
kis bbox create -w	-w / --workers	4	More than one input only; ignored for a single file
kis rules -w	-w / --workers	1	Fully serial; clamped to the number of resolved inputs
kis rules --mode doc	—	—	No workers flag; the whole corpus becomes one document

Workers multiply peak memory

kis rules --workers N holds N fully analysed documents at once. Every page’s words, merged blocks, lines, tables and spatial index, with no page, size or time cap. --mode doc merges the entire corpus into one document, so peak memory is the whole corpus regardless.

Aggregated page dimensions are an arithmetic mean

Document-level statistics average page width and height across every page with non-zero dimensions. For a document mixing portrait and landscape, or letter and legal, that aggregate is physically meaningless, and it is what document-level percentage queries scale by.

Errors

Message	Cause
Error: '<command>' is not installed.	The rules plugin binary is absent from every search location, or present without an executable bit
failed to stat input path	An input does not exist
loading hocr failed from given input file	The path could not be opened, or the HTML parser failed
failed to create output directory	-o could not be created
kis bbox create: no input files found to process.	No input matched *.hocr or *.hocr.html; exit non-zero
refusing to overwrite input <path>: output path resolves to the same file (pass a different -o)	The output would replace the input, as for a .bbox file given as input
unknown --format "<value>" (want auto, hocr)	--format names an input format the command does not read
writing json to output file path failed	Permissions, or a full disk
<n> of <m> hocr files failed to convert (first: <path>: <error>)	One or more inputs failed; the others were converted

The full table is in Error reference; diagnosis steps are in Troubleshooting.

Continue with

• Document model: what pages, lines, words and boxes mean
• Extraction: fields, checkboxes and regions
• Tables: table and master-detail detection
• Matching and confidence: fuzzy matching and OCR confusion handling
• Rules over documents: reaching a document from a rule
• CLI reference: kis rules, every flag and mode