Document Recipes

Everything below is backed by lib-doc. One engine reads bytes into a faithful structured document and writes them back out, in any supported format, with tiered extraction (cheap and deterministic first; OCR/AI only when earned) and provenance/quality on everything. Input is a file path; output goes to a file or stdout (status and warnings go to stderr, so pipes stay clean).

Quick start

kis doc convert report.docx report.md          # convert between formats
kis doc convert notes.md notes.pdf             # markdown → PDF
kis doc read invoice.pdf                        # parse and print as Markdown
kis doc read scan.pdf --quality                 # + show the extraction quality report
kis doc text contract.docx                      # plain-text (pre-chunking) extract
kis doc tables statement.pdf                     # pull out tables as CSV
kis doc classify mystery-file                    # what format / document type is this?
kis doc tools list                               # what OCR engines / converters are installed

Format is auto-detected from the bytes (and filename). Output format is inferred from the output file’s extension. Override either with --from / --to.

Supported formats

kis doc formats

format	read	write	notes
Markdown	✅	✅	GFM tables, task lists; near-lossless round-trip
HTML + CSS	✅	✅	the universal intermediate
DOCX	✅	✅	headings, nested lists, tables, runs, hyperlinks, images
XLSX	✅	✅	preserves cell types, formulas, merges, named ranges
PPTX	✅	—	slides → sections (title, body, tables, speaker notes)
PDF	✅	✅	tiered: text layer → layout → OCR → vision
CSV / TSV	✅	✅	delimiter auto-detected
plain text	✅	✅	—
images (PNG/JPEG/…)	✅	—	OCR / vision, or preserved as a figure

Exotic/legacy formats (e.g. .epub, .rtf, .doc) are reachable through an external converter, see --via.

convert

kis doc convert <input> <output>. Read input, write output. Formats are inferred from extensions.

kis doc convert report.docx report.md
kis doc convert deck.pptx deck.md
kis doc convert data.xlsx data.csv
kis doc convert page.html page.pdf
kis doc convert notes.md notes.docx --theme brand.json     # apply a theme on write
kis doc convert in.pdf out.md --from pdf --to markdown      # force formats

It prints what it did and, for lossy conversions, the extraction confidence:

converted report.docx (docx) → report.md (markdown), 4128 bytes

Batch a directory (1:1)

Convert many files at once with --input (a directory) and an optional --pattern (a glob on the file name). Each file is converted 1:1; --to sets the target format.

# every PDF in ./docs → a .md next to it
kis doc convert --input ./docs --pattern '*.pdf' --to md

# recurse into subdirectories
kis doc convert --input ./docs --pattern '*.docx' --to md -r

# write results to a separate tree, mirroring the input's structure
kis doc convert --input ./docs --to html --output ./out -r

# all recognized files (no --pattern), e.g. a mixed customer folder, 8 in parallel
kis doc convert --input ./customer-files --to md --output ./extracted -r -w 8

By default each output lands next to its input (same directory, extension swapped). With --output <dir> the input’s subdirectory structure is mirrored under it. Unrecognized files are skipped; a basename clash (e.g. a.html and a.md both → a.html) is reported, not silently overwritten. All the read-side flags (--ocr, --render, --force-ocr, --via, --theme, --max-tier) apply to every file in the batch.

flag	meaning
-i, --input <dir>	input directory (triggers batch mode)
--pattern <glob>	match file names, e.g. '*.pdf' (default: all)
-O, --output <dir>	output directory; mirrors input structure (default: alongside inputs)
-r, --recurse	descend into subdirectories
-w, --workers <n>	files converted in parallel (default 1)
--to <fmt>	target format (required for batch)

--workers (default 1) controls parallelism: in batch it’s how many files convert at once; for a single file it’s how many PDF pages process at once (rasterize + OCR across cores). In batch, each file’s pages run serially so total concurrency stays ~--workers.

read

kis doc read <file>, parse a document and print it. Default output is Markdown; switch with a flag.

kis doc read invoice.pdf                 # → Markdown on stdout
kis doc read invoice.pdf --text          # → plain text
kis doc read invoice.pdf --json          # → the Document model as JSON (for tooling)
kis doc read invoice.pdf --quality       # print the quality report (to stderr) too
kis doc read invoice.pdf -o invoice.md   # write to a file instead of stdout

--json emits the full structured model (blocks, inlines, styles, provenance, per-page quality): useful for piping into other tools or inspecting extraction.

classify

kis doc classify <file>, detect the format and document type without a full parse.

kis doc classify "Closing Disclosure.pdf"
# format:  pdf
# doctype: contract

Doctypes: prose, form, contract, invoice, receipt, slide, spreadsheet, report, article, code. (Heuristic; a host can inject a trained classifier.)

quality

kis doc quality <file>, print the extraction quality report: overall and per-page confidence, the extraction method/tier per page, and any flags.

kis doc quality scan.pdf
# overall confidence: 0.90
# flags: missing_text_layer
#   page 1: conf=0.90 method=ocr tier=ocr
#   page 2: conf=0.88 method=ocr tier=ocr

Flags worth knowing: missing_text_layer (scanned), parse_fallback (the text layer was garbled and got OCR’d), low_conf_ocr, mixed_tiers (some pages born-digital, some scanned), degraded_scan.

tables

kis doc tables <file>, extract every table.

kis doc tables statement.pdf            # CSV rows, grouped per table
kis doc tables statement.pdf --md       # pipe-separated instead of CSV

text

kis doc text <file>, plain-text flatten (the pre-chunking normalization the RAG stack uses).

kis doc text contract.docx
kis doc text scan.pdf | wc -w

formats

kis doc formats
# read:  csv docx html image markdown pdf pptx text xlsx
# write: csv docx html markdown pdf text xlsx

Scanned & garbled PDFs (OCR)

PDFs come in three flavors, handled automatically:

1. Born-digital (real text layer): extracted directly, no OCR. Fast.
2. Scanned (no text layer): pages are rasterized to images and OCR’d.
3. Garbled (a text layer that decodes to mojibake: common with subset/CID fonts): detected and routed to OCR instead of emitting garbage.

When tesseract and Poppler (or Ghostscript) are installed, all three just work, kis doc uses them automatically and only on the pages that need them, so clean PDFs stay fast:

kis doc tools install tesseract poppler      # one-time
kis doc convert scanned.pdf scanned.md       # auto-rasterize + OCR the scanned pages
kis doc read garbled.pdf --quality           # garbled layer → OCR, flagged parse_fallback

Controls:

--ocr auto|tesseract|paddleocr|none     # OCR engine (default auto: tesseract if installed)
--render auto|poppler|ghostscript|none  # PDF→image engine (default auto: poppler, then gs)
--force-ocr                              # ignore the text layer; rasterize + OCR every page
--max-tier text_layer|structural|ocr|vision   # cap escalation (e.g. forbid OCR cost)
-w, --workers N                          # OCR/rasterize N pages in parallel (default 1)

Examples:

kis doc read scan.png --ocr tesseract                 # OCR an image
kis doc convert messy.pdf out.md --force-ocr          # re-OCR everything (bad text layer)
kis doc convert big.pdf out.md --render ghostscript   # use Ghostscript to rasterize
kis doc read clean.pdf --ocr none                     # never OCR (text layer only)

OCR runs per page, up to --workers (-w) pages in parallel (default 1; bump it for big scans), a vision LLM (Tier 4) can also be wired by a host for the hardest pages; the CLI uses the deterministic + OCR tiers.

External converters (--via)

For exotic formats, or when another tool simply does better, route the read side through an external CLI converter. lib-doc runs it once, captures its Markdown/ HTML, and parses that back into the model.

kis doc convert book.epub book.md --via markitdown      # Microsoft markitdown
kis doc convert old.docx old.md --via pandoc            # pandoc
kis doc convert sheet.xlsx sheet.md --via libreoffice   # headless LibreOffice

Install them with kis doc tools install markitdown (etc.). A missing tool gives a clear error, never silent garbage.

Managing tools (kis doc tools)

The OCR engines and converters are external programs. List and install them (OS-aware: Homebrew on macOS; apt/dnf/pip on Linux):

kis doc tools list
kis doc tools install all              # install everything missing
kis doc tools install tesseract        # install one
kis doc tools install poppler          # PDF rasterization (pdftocairo/pdftoppm)
kis doc tools install ghostscript      # alternative PDF rasterizer (gs)
kis doc tools install markitdown       # any-format → Markdown (Python; isolated via pipx)
kis doc tools install paddleocr        # alternative OCR engine

list shows what’s installed, the version, and which feature each enables:

✓ tesseract    tesseract 5.5.2   Offline OCR engine (scanned PDFs and images)
✓ poppler      pdftocairo 26.04  PDF → image rasterization
✗ markitdown   missing           Microsoft's any-format → Markdown converter

Notes:

• tesseract language packs: brew install tesseract-lang (macOS) / apt install tesseract-ocr-<lang> (Linux).
• Python tools install into isolated environments (pipx) or, where a library must be importable (paddleocr), into user site-packages.
• Tools installed off the default PATH (e.g. pipx’s ~/.local/bin) are still found and used.

Theming on create

The write path applies a theme (fonts, palette, heading styles, page margins, brand-voice authoring rules). Pass a JSON theme file:

kis doc convert notes.md notes.docx --theme brand.json
kis doc convert notes.md notes.pdf  --theme brand.json

A theme is the lib-doc Theme shape, e.g.:

{
  "Name": "kis.ai",
  "Fonts": { "Body": "Inter", "Heading": "Inter", "BaseSizePt": 11 },
  "Palette": { "Text": "#0a1f44", "Primary": "#1456c4" },
  "Heading": [ { "SizePt": 24, "Bold": true, "Color": "#0a1f44" } ]
}

The same document renders under different themes, content and presentation stay separate.

How it works

• One model. Every reader normalizes to a structured Document; every writer generates from it; conversion routes through it. Reading order, tables, lists, and provenance are preserved, not flattened to “get the text out”.
• Tiered PDF extraction. text layer → layout analysis (multi-column reading order) → OCR (scanned) → vision (hardest), a clean PDF never touches a model.
• Parallel. Per-page rasterize/OCR/assembly runs across all CPU cores; pages are reassembled in order.
• Honest quality. Low-confidence or garbled extractions are flagged in the quality report, never passed off as clean.

Troubleshooting

PDF → Markdown gives gibberish (symbols, #$<>& soup). The PDF’s text layer is garbled (subset fonts). Install OCR tools and re-run; kis doc will detect it and OCR automatically:

kis doc tools install tesseract poppler
kis doc convert file.pdf out.md            # now OCR'd
kis doc convert file.pdf out.md --force-ocr  # if a partial/odd text layer slips through

“this PDF has no usable text layer; install OCR tools…”: exactly that: run the suggested kis doc tools install …, then retry.

Output is empty for a scanned PDF. No OCR engine was available. Install tesseract + a rasterizer (poppler or ghostscript).

Small/dense scans OCR poorly. Increase rasterization DPI, wire a higher-DPI rasterizer in a host, or use --render ghostscript which often renders cleaner.

Flag reference

Common across read / convert (and where noted):

flag	default	meaning
--from	detect	source format override
--to	from output ext	target format (convert)
--ocr	auto	OCR engine: auto/tesseract/paddleocr/none
--render	auto	PDF rasterizer: auto/poppler/ghostscript/none
--force-ocr	off	ignore the text layer; rasterize + OCR every page
-w, --workers	1	parallel workers (batch: files; single: PDF pages)
--max-tier	vision	cap extraction: text_layer/structural/ocr/vision
--via	—	external converter: markitdown/pandoc/libreoffice
--theme	—	JSON theme file (convert/write)
--json	off	output the Document model as JSON (read)
--text	off	output plain text (read)
--quality	off	print the quality report (read)
-o, --out	stdout	write to a file (read)
--md	off	pipe-separated table rows (tables)