Data-Driven Testing

Run the same test once per row of tabular data. Both runners support this with slightly different syntax.

v2 (`kis test run`): `data:` + `table:`

Declare named data sources at the suite level, reference one from a test:

# suite.yaml (or the test file of a single-file suite)
data:
  users:
    type: csv
    file: fixtures/users.csv        # relative to the declaring suite's directory
  scopes:
    type: inline
    rows:
      - { name: full,    scope: "read write" }
      - { name: readonly, scope: "read" }

tests:
  - name: get-user
    table: users                    # once per CSV row
    steps:
      - name: fetch
        http:
          url: "{{baseurl}}/users/{{id}}"
          method: GET
        assertions:
          - status_code == 200
          - response.name == "{{name}}"

Source types

`type`	Source	Notes
`inline`	`rows:`: a YAML list of maps	no file needed
`yaml` / `yml`	`file:`	top-level list of maps, or `{ rows: [...] }`; `path:` selects a nested array
`json`	`file:`	same shapes as yaml; `path:` selects a nested array
`jsonl` / `ndjson`	`file:`	one JSON object per line; blank lines skipped
`csv`	`file:`	first row = header; all values are strings; leading space trimmed; ragged rows are an error

Selecting rows from inside a document (`path:`)

API dumps rarely put the rows at the top level. path: walks to the array, with dotted fields and [n] indexes:

data:
  users:
    type: json
    file: fixtures/api-dump.json
    path: data.users              # {"data": {"users": [ ... ]}}
  first_batch:
    type: json
    file: fixtures/api-dump.json
    path: results[0].items
  from_yaml:
    type: yaml
    file: fixtures/nested.yaml
    path: data.users

The selected node must be an array of objects; anything else is an error naming what was found. A field that isn’t there lists the ones that are (no field "customers" (have: users)).

path: applies to yaml and json only: csv, jsonl and inline have no envelope to select from, and setting it there is an error rather than a silent no-op.

JSONL

data:
  events:
    type: jsonl
    file: fixtures/events.jsonl   # {"id": 10, ...}\n{"id": 11, ...}

One object per line. Blank lines are skipped, so a trailing newline or a blank separator is fine. A malformed record names its line number: what you need to find it in a large dump. Records are capped at 8MB each.

Dynamic tables: rows fetched by a step

table: also accepts a {{variable}} reference, taking its rows from scope instead of a declared source. That is how one step’s output becomes another test’s table:

before_all:
  - name: fetch-user-list
    http:
      url: "{{baseurl}}/users"
      method: GET
      response:
        variables:
          user_rows: "jq: .response.users"    # an array of objects

tests:
  - name: per-user
    table: "{{user_rows}}"                    # one test per fetched user
    steps:
      - name: get
        http:
          url: "{{baseurl}}/users/{{id}}"
        assertions:
          - status_code == 200
          - response.name == "{{name}}"

The braces are what selects this path, so a data: source and a variable may share a name without either shadowing the other. A value that merely contains a template (table: "users_{{env}}") is a computed source name, not a dynamic table.

Accepted values: an array of objects, an already-typed row list, or a JSON array still in string form (what test.set_variable often holds). Anything else fails with what it found.

Scope is the constraint worth understanding. The rows must be visible where the test starts, which means a before_all hook (its writes persist into the suite scope) or a suite/env variable. A variable set inside another test does not carry across: each test starts from a snapshot of the suite scope and its mutations stay local, by design. Steps in the same test can share variables freely, but iteration is per test, so a table has to be resolvable before the test’s first step runs.

Semantics

file: paths resolve relative to the directory of the suite that declares the source.
Lookup walks the suite tree leaf → root, so a parent suite can declare tables for all its descendants.
Rows lazy-load on first use and are cached for the run.
A table’s columns are every key its rows use. A row that leaves one out has it empty, as an empty CSV cell is, unless the test or its suite gives the name a value; a name that is no column of the table stays undefined, so a misspelled reference is still reported.
Each row clones the test; row fields override test.variables (and anything below them in the scope precedence).
Per-iteration result names take a suffix from the first present of: _name column → name column → id column → [row=N].
Iteration is sequential and stops at the first failed row unless the test sets continue_on_error: true.
A table: reference that yields zero rows marks the test errored (an empty fixture is treated as a bug, not a skip).
before_each/after_each hooks run around every row.

Legacy (`kis test tests`): `tables:` + `table:`

File-level tables: with per-step (or per-testcase) table: references:

tables:
  users:
    type: csv
    file: users.csv           # relative to the suite directory

steps:
  - name: get-user-by-id
    table: users
    rest:
      url: "{{baseurl}}/users/{{id}}"
      method: GET

Table types: csv (optional separator:, default ,) and excel (requires sheet:).
File-level tables: propagate automatically to any step/testcase/scenario/testplan in the file that references a table: without declaring its own.
Rows show up in the summary as -RC0, -RC1, … suffixes, only when there are ≥ 2 records; single-record runs keep clean names.

CLI data files (`-d/--datapath`)

The tests runner also takes data on the command line, repeatable and merged in order:

kis test tests -p examples/v1/04-data-driven -d examples/v1/env.json -s
kis test tests -p tests/ -d vars.yaml -d users.csv -s

.json / .yaml files merge into the variable map (later files override earlier).
.csv files fan out: every row becomes a separate execution record over the merged variables.
-v key=value inline vars override everything from -d.

Choosing an approach

Fixture-shaped inputs that belong with the tests → v2 data:/table: (checked in next to the suite, resolved relative to it).
Environment-shaped values (URLs, credentials) → env files, not tables (CLI Reference).
Quick ad-hoc parameter sweeps with the legacy runner → -d file.csv.