Tables and repeating structures

Repeating structure — tables, grid sections, line items — is recovered from geometry alone. There is one detection algorithm: text-alignment clustering over OCR blocks. Nothing reads ruled lines, cell borders or an embedded table model, so a borderless table and a ruled table are detected identically, and a table drawn entirely with rules but misaligned text is not detected at all.

Read The document model first for pages, blocks, lines and coordinates, and Extracting fields for the anchor and schema machinery that this page builds on.

Table extraction reaches both surfaces. The kis CLI runs it over local .bbox files; rules.svc runs it over a document sent with the execute-rule request. Detection, the returned shapes and the rule-facing calls are identical on both — a rule set that extracts a table under kis ocr extracts the same table over HTTP.

What is reachable from where

Capability	Rule (.grl)	kis CLI	rules.svc
Detected tables, from Analyze()	yes	kis bbox tables	yes
Schema-selected tables (x-table)	yes	kis ocr	yes
Column-aligned grid sections	yes	kis ocr	yes
Label-driven repeating rows	yes	kis ocr	yes

Detection

Table detection runs during Analyze() on every document that loads, on both surfaces, and fills Document.Tables. There is no switch that turns it off. The detector walks the document’s lines in one pass and grows a region while the next line has a similar block count (within one) or aligns on column x-positions, and the vertical gap stays under the gap ceiling. A region that ends up with fewer than the minimum rows or columns is discarded.

Setting	Value on the Analyze() path
Minimum rows	2
Minimum columns	2
Row-gap ceiling	2.0 × the document’s median line gap
Column x-alignment tolerance	10.0 px

Those four values are fixed. Analyze() builds the detector with them on every run, and neither the CLI nor the service exposes a flag, a config key or a rule call that changes any of them — a region under 2 rows or 2 columns is discarded on every document, on both surfaces.

The gap ceiling is what stops a table at a page break, and only incidentally — the detector never compares page numbers. Page-local coordinates usually make the gap between the last line of one page and the first line of the next enormous, so a table that continues onto the next page is detected as two tables, each carrying the page of its own first line. When a page’s last line sits high enough that the page-local gap stays under the ceiling, the region runs straight through the break and you get one table labelled with the earlier page. There is no table stitching either way.

Output shape

Type	Fields
Table	Rows, Columns, Box, HeaderRows, Page, Confidence, TableScore
Row	Index, Cells, Box, Height, Confidence, IsTotal, IsSeparator, IsRepeatedHeader
Cell	Row, Col, RowSpan, ColSpan, Content, Text, Box, IsHeader, Confidence, IsMerged
Column	Index, Header, Box, Width, Alignment, Confidence

Column.Alignment is one of AlignUnknown, AlignLeft, AlignRight, AlignCenter.

Caution

Analyze() fills only a subset of those fields. Cell.RowSpan and ColSpan are always 1, IsMerged is always false, and Row.IsTotal, IsSeparator and IsRepeatedHeader are always false: the pass that infers spans from box geometry and tags total, separator and repeated-header rows is table.DetectWithSpec, which no rule, CLI verb or extractor calls. Table.TableScore and the Confidence on Row, Cell and Column are declared and never written — they are zero on every table you will see. Table.Confidence is the only populated score.

Reading a table from a rule

Get the document by the name it was loaded under, then take a table by index or by an anchor phrase. The Safe* entry points wrap a missing table instead of returning nil — all except SafeTablesBetween, which is a plain nil on a miss.

Call	Returns	Selection
SafeTable(index)	SafeTableResult	0-based index into Document.Tables
SafeTableBelow(anchor)	SafeTableResult	nearest table below the anchor; falls back to the first table on a later page
SafeTableAbove(anchor)	SafeTableResult	nearest table above the anchor; falls back to the last table on a prior page
SafeTableBetween(start, end)	SafeTableResult	first table in reading order between two anchors
SafeTableBelowFuzzy(anchor, perWord, total)	SafeTableResult	as above, tolerating OCR damage in the anchor
SafeTableAboveFuzzy(anchor, perWord, total)	SafeTableResult	”
SafeTableBetweenFuzzy(start, end, perWord, total)	SafeTableResult	”
SafeTablesBetween(start, end)	[]*TableResult	every table between two anchors — unusable from a rule, see below
TableCount() / HasTables()	int64 / bool	count and presence

Anchor matching for all of these is case-insensitive and takes the first match of the phrase. SafeTableBetween and SafeTablesBetween return nothing when the two anchors resolve out of reading order.

Once you hold a SafeTableResult, these methods are nil-safe and callable from a rule:

Method	Returns
RowsAsMaps()	[]map[string]string — one map per data row, keyed by column header
AsMap()	map[string][]string — one entry per column
RowCount() / DataRowCount() / ColumnCount()	int
HasData() / IsEmpty()	bool

Headerless columns get the synthetic keys col_0, col_1, … in RowsAsMaps and AsMap. Empty cells are present as empty strings.

rule ExtractLineItems salience 100 {
    when
        bbox.Has("doc")
    then
        out.Set("line_items",
            bbox.Get("doc").SafeTableBelow("Invoice Items").RowsAsMaps());
        Retract("ExtractLineItems");
}

Guard on bbox.Has(name), not on the document itself. bbox.Get on a name that was never loaded returns nothing, and the next call in the chain dereferences it — that panic aborts the whole execution and discards all output.

Caution

The per-cell accessors — Cell(row, col), DataCell(row, col), CellByHeader(row, header), Row(i) and DataRow(i) — take Go int arguments. A rule’s integer literals are int64, so SafeTable(0).Cell(1, 1) aborts the execution with reflect: Call using int64 as type int. They only dispatch when every index comes from another call that already returns int — RowCount(), DataRowCount() or ColumnCount(). Read whole rows with RowsAsMaps() or whole columns with AsMap() instead.

Caution

RowsAsMaps() returns []map[string]string. The rule array helpers only accept []map[string]any, so passing table rows to array.Len or array.FindRowBy aborts the execution; pass them straight to out.Set. Grid-section rows (below) are []map[string]any and do work with those helpers. SafeTablesBetween returns []*TableResult, which no rule helper accepts and whose element methods are all unsafe, so there is nothing a rule can do with the result. Select tables one at a time with SafeTableBelow or SafeTable.

Caution

Only the methods declared on SafeTableResult are nil-safe. Every other table method — Headers(), ColumnByName(), ColumnData(), FindRow(), IsColumnRightAligned() and the rest — is promoted from the unsafe wrapper and panics when the anchor matched no table. A panic in one rule aborts the whole execution and discards all output. Guard with HasData() or use RowsAsMaps(), which returns nil.

Selecting a table from a schema

A .schema file whose top-level type is array and which carries x-table is a table target. The filename without its extension is the name rules invoke. Register schemas with --schema or --schemas on kis ocr, or send them under bbox.__schemas on an execute-rule request to rules.svc, then call bbox.Get("doc").ExtractTable(name) — or SafeExtractTable(name), which returns an empty slice rather than nil.

Key	Type	Meaning
x-table.containing	string	first table with a cell containing this text, case-insensitive
x-table.anchor_below	string	first table below this phrase
x-table.anchor_above	string	first table above this phrase
x-table.between	[start, end]	first table between two phrases; both must be non-empty
x-table.index	int64, 1-based	table by ordinal
x-table.fuzzy.max_distance	int64	per-word edit distance for anchor_below / anchor_above / between
x-table.fuzzy.max_total	int64	total edit distance across the anchor phrase

Precedence is fixed and does not depend on file order: containing, then anchor_below, then anchor_above, then between, then index. Setting two locators means the lower one is ignored.

{
  "type": "array",
  "title": "Invoice line items",
  "x-table": {
    "anchor_below": "Invoice Items",
    "fuzzy": { "max_distance": 1, "max_total": 2 }
  },
  "items": {
    "type": "object",
    "properties": {
      "description": { "type": "string" },
      "amount":      { "type": "string", "x-anchor": "Amount Due" }
    }
  }
}

Rows come back as []map[string]any keyed by the items property names. A property maps to the detected column whose header equals the property name; declare x-anchor on the property when the printed header differs from the key you want. The header comparison is exact — no fuzz, no case folding. Properties that find no column, and properties whose cell is empty, are absent from the row map; a row with no matched properties is dropped. x-derive runs per row after the columns are mapped.

Caution

x-separator is not a general repeater control. It is read only on an array schema that has no x-table — add x-table and it is never parsed at all. On that separator-only path, just table_row and the empty value do anything, and both re-route to table detection using x-region.start_anchor (with end_anchor, when present) as the table locator. Any other value returns nil. x-max-records and x-min-match are parsed and then never consulted.

Column-aligned grid sections

ExtractColumnAlignedSection is the extractor for columnar forms where the numbers line up under printed headings but the detector’s row clustering is too coarse — closing statements, fee schedules, premium breakdowns. It resolves column x-positions from header labels, takes the vertical band between two anchors, and assigns every block in the band to a column by nearest centre X.

Two call shapes reach the same code:

// Nil-safe wrapper — returns nil if no document is loaded under that name.
out.Set("charges", bbox.ExtractColumnAlignedSection(
    "page",
    "Origination Charges",
    "Services Borrower Did Not Shop For",
    "At Closing,Before Closing,Paid By Others"));

// Chained document method — equivalent, but bbox.Get must have matched.
out.Set("charges", bbox.Get("page").ExtractColumnAlignedSection(
    "Origination Charges",
    "Services Borrower Did Not Shop For",
    "At Closing,Before Closing,Paid By Others"));

Pass "" as the section end to run to the bottom of the page.

The columnHeaders mini-DSL

columnHeaders is one comma-separated string. Each entry is either a column spec or an @ directive.

Column spec	Effect
Name	text mode: find the header by fuzzy phrase search, use its centre X
N:Name	position mode: explicit centre X in pixels, box is N ± 25px
Name=>Alias	search OCR for Name, emit the value under Alias
Name#N	pick the Nth occurrence when the header text repeats (1-based)
Parent<dir>|Child	chain: navigate from Parent in dir to find Child; dir is below, above, right or left

Chains take any number of hops (A<below>|B<right>|C), and a => rename attaches to the last segment. Chain and plain text specs mix freely.

Directive	Effect	Default
@description=>Key	rename the built-in description output key	description
@row_number=>Key	rename the built-in row-number output key	row_number
@desc_split=>DELIM=>Field	split the description on DELIM; the right-hand part becomes Field	none
@rogue=>chars	strip these characters from the description and every column value	none

@desc_split is repeatable and directives apply in the order given. @rogue is applied first, before row-number extraction and description splitting.

Each returned row is a map[string]any with description, row_number, one key per column alias and one key per @desc_split field. Missing values are present as empty strings.

out.Set("charges", bbox.ExtractColumnAlignedSection("page",
    "Origination Charges", "Services Borrower Did Not Shop For",
    "@description=>FeeName,@desc_split=>to=>Vendor,Borrower-Paid<below>|At Closing=>BPAC,Seller-Paid<below>|At Closing=>SPAC,Paid By Others"));

Grid failure modes

Every failure returns nil. Call Toggledebug() on the document to see which stage failed — it prints the resolved column positions, the description boundary and the Y band to stderr.

Symptom	Cause
nil result	columnHeaders empty, no non-@ spec left, no column position resolved, the start anchor was not found, or no blocks fell in the band
Rows from the wrong part of the page	the section end anchor was empty or unresolvable, so the band ran to the page bottom — there is no signal distinguishing the two
A column silently disappeared	position mode is decided by the first column spec alone (@ directives are stripped first); if it looks like N:Name, every spec is parsed that way and plain-text and chain specs are dropped
A stray mark appended to the last column	any block at or right of the description boundary goes to the nearest column, with no maximum distance and no unassigned bucket; values accumulate space-joined
row_number empty on numbered rows	the split regex requires exactly two digits followed by whitespace — 01 FEE splits, 1 FEE and 100 FEE do not
@desc_split did nothing	the delimiter is matched with a space on each side; FEE to VENDOR splits, FEEtoVENDOR does not
@rogue stripped the wrong things	the value is used verbatim as a character set — write the actual characters, not escape sequences
#N picked an unexpected header	occurrences are ordered top-to-bottom first, then left-to-right, so headers on different lines are not ordered left-to-right

Caution

The DSL is split on commas before anything else is parsed, and there is no escape mechanism. A header, alias, @desc_split delimiter or @rogue character set containing a comma cannot be expressed.

Post-processing rows

bbox.PostprocessRows(rows, key, fn) applies one transform to the string at key in every row. It mutates in place and returns the same slice, so call it repeatedly to chain. Rows where the key is absent or not a string are skipped, and an fn the registry does not know returns the rows untouched.

fn	Effect
trim	strip leading and trailing whitespace
upper / lower	case conversion
normalize_spaces	collapse runs of whitespace
trim_currency	strip $ and ,
ExtractRegex("pattern", "defaultVal")	joined non-empty capture groups, or the whole match when the pattern has no groups, or defaultVal on no match

ExtractRegex is one string containing quotes, so a rule has to escape them: "ExtractRegex(\"([0-9.]+)\", \"\")". An unparseable pattern returns the rows untouched.

Chain through memory, one in.MemSet per transform:

in.MemSet("charges", bbox.ExtractColumnAlignedSection("page", "Origination Charges", "", "At Closing"));
in.MemSet("charges",
    bbox.PostprocessRows(in.MemGetRows("charges"), "At Closing", "trim_currency"));
out.Set("charges", in.MemGetRows("charges"));

Caution

Do not chain through a local rule variable. rows = bbox.ExtractColumnAlignedSection(…); rows = bbox.PostprocessRows(rows, …); aborts the execution with reflect: Call using zero Value argument, for every fn including the named ones. Only two forms dispatch: the fully inline bbox.PostprocessRows(bbox.ExtractColumnAlignedSection(…), …), and the in.MemGetRows form above.

Repeating rows from a rule

RepeatingExtractingFields reads label-driven repeating records off the document’s lines. A record starts on every line containing the first label; each label’s value runs to the next label or to an end marker.

out.Set("borrowers", bbox.Get("doc").RepeatingExtractingFields(
    "Borrower Name,Account,Balance",  // labels, comma-separated
    3,                                 // max lines per record
    4,                                 // max words per value
    "Total,Subtotal",                  // end markers, comma-separated
    "Account:1",                       // per-label word caps, label:n
    "Balance"));                       // labels whose value sits below, comma-separated

Parameter	Meaning
labels	comma-separated, ordered; the first is the record-start marker
max lines	lines a record may span; 0 makes every record a single line
max words	default word cap per value; 0 means no cap
end markers	comma-separated phrases that terminate a value
per-label word caps	label:n pairs, comma-separated
vertical labels	comma-separated labels whose value is the nearest block below, not to the right

Records come back as []map[string]any keyed by label. A record is kept only when at least half its labels produced a value; that ratio is fixed and not exposed. Label matching is a case-sensitive substring test against the joined line text, so Account also matches Account Number.

Caution

A vertical label’s value is searched across every block in the document, with no page check and no distance cap. On a merged multi-page document the nearest horizontally-overlapping block below can come from a different page.

What happens at a page boundary

Coordinates are page-local, so every Y band exists on every page. Each extractor resolves that differently, and nothing stitches a table, a grid section or a repeating row across the break.

Extractor	Page behaviour
Table detection	never checks the page; the large page-local gap at a break usually ends the region, so a continuing table becomes two tables — but a small gap merges them into one
ExtractColumnAlignedSection	single page by design: blocks are filtered to the start anchor’s page. Rows continuing onto the next page are not returned
Checkbox detection	page-scoped: identical page-local boxes on different pages stay distinct; overlapping boxes on one page collapse to the higher-confidence one
RepeatingExtractingFields vertical labels	no page check — can pull a value from another page

To work across pages at all, load the pages as one document. kis ocr doc merges every input .bbox into a single multi-page document and renumbers pages sequentially from 1; kis ocr page runs each page as its own document. Over HTTP the choice is yours at the call site: one pages array carrying every page is one document, and one entry per page is one document each.

From the CLI

kis bbox tables invoice.bbox -f json
kis bbox table -a "Invoice Items" -H "Item" -H "Price" invoice.bbox
kis ocr doc -r ./rules ./pages --schemas ./schemas --bbox-name doc

Command	Purpose
kis bbox tables	detect and print every table; -i/--input repeatable, -f/--format text|json|csv, -o/--output
kis bbox table	print the table nearest an anchor; -a/--anchor required, -H/--header repeatable
kis ocr page / kis ocr doc	run rules over pages or a merged document; -r/--rules required, --schema, --schemas, --bbox-name

kis bbox table -H fails the command when an expected header is missing, which makes it a usable regression check on a fixture set. Header comparison is case-insensitive and ignores surrounding whitespace. It reports table has no headers when the detector found no header row, and missing headers: <list> (found: <list>) otherwise. When the anchor matches nothing it prints No table found near anchor: <anchor> and still exits 0.

Note

kis bbox and kis ocr are proxy commands that run a separately installed plugin. If it is not installed you get Error: 'bbox' is not installed. / Error: 'ocr' is not installed. — the command name you typed is interpolated. See the CLI reference.

From the service

rules.svc runs the same extraction on the same route. The execute-rule request carries the document alongside the facts under the reserved bbox key: __docs maps each name a rule passes to bbox.Get to that document’s word-box JSON, and __schemas carries x-table schemas by the name a rule passes to ExtractTable. Everything else about the call is unchanged — name selects the rule set, every other top-level key is a fact, and the response is what the rules wrote to out.

{
  "name": "invoice-tables",
  "bbox": {
    "__docs": {
      "doc": {
        "pages": [
          {
            "number": 1,
            "width": 612,
            "height": 792,
            "words": [
              { "text": "Invoice", "box": {"x0": 72, "y0": 100, "x1": 130, "y1": 115}, "confidence": 0.98, "page": 1 }
            ]
          }
        ]
      }
    },
    "__schemas": {
      "line_items": {
        "type": "array",
        "x-table": { "anchor_below": "Invoice Items" },
        "items": { "type": "object", "properties": { "description": { "type": "string" } } }
      }
    }
  },
  "currency": "EUR"
}

Drop __schemas and send the document map directly under bbox when no schema is in play.

The document names are the ones the rule already uses, so the rule text is the same one you run under kis ocr doc --bbox-name doc:

rule ExtractLineItems salience 100 {
    when
        bbox.Has("doc")
    then
        out.Set("line_items",
            bbox.Get("doc").SafeTableBelow("Invoice Items").RowsAsMaps());
        Retract("ExtractLineItems");
}

RowsAsMaps() lands in the response under line_items. The word-box JSON is the format described in OCR ingestion — run the recogniser and any .bbox conversion before the call; the service ingests OCR output, it does not produce it. Headers, tenancy and the error envelope are on the service page.

Caution

Every trap on this page applies unchanged over HTTP. A method that panics on a missed anchor, an int64 literal passed to Cell(row, col), or a []map[string]string handed to array.Len aborts the whole execution — and the service returns 400 with the run’s entire output discarded, so a single bad rule loses the tables the other rules already extracted.

Confidence

Every confidence on this page is a hand-tuned heuristic, not a calibrated probability. Table.Confidence is the mean of two factors — how little the per-row cell count varies, and a step function of the table’s size (0.5 under 3 rows, 0.8 from 3×2, 1.0 from 5×3). Column alignment is detected but does not feed it. There are no cell, row or column confidences, and grid rows carry none at all. Only ExtractFormScored reports per-field confidence and coverage, and it does not apply to tables, grid sections or repeating rows. Route review on your own thresholds, and treat a score as an ordering, not a probability.

Continue with

• The document model — pages, blocks, lines, coordinates
• Extracting fields — anchors, schemas, scored results
• Fuzzy matching — the OCR-aware distance these anchors use
• Documents in rules — loading a document and reaching it from a rule
• CLI reference — kis bbox and kis ocr in full
• The rules service — the execute-rule route, headers and errors