Assertions & Variables

How the v2 runner (kis test) evaluates assertions, resolves {{variables}}, and passes data between steps. For the legacy response.validate expressions used by test.svc tests, see the legacy tests mode.

Assertions

Each entry under a step’s assertions: is one expression:

assertions:
  - status_code == 200
  - response.id != null
  - duration_ms < 500
  - response.items.length > 0
  - response.email contains "@kis.ai"
  - response.type starts_with "user."
  - headers["content-type"] contains "application/json"
  - response.name == "{{username}}"
  - "jq: [.response.users[].id] | length > 2"

All assertions in a step are evaluated (no short-circuit); any failure fails the step. Failures report expected vs. actual per expression.

Grammar

<lhs-path> <operator> [<rhs>]        # comparison form
jq: <jq expression>                  # free-form jq; truthy output passes

Binary operators: ==, !=, <, <=, >, >=, contains, not_contains, starts_with, ends_with, matches (Go regexp), includes (array alias of contains), has_key, not_has_key.

Unary operators (no RHS): is_string, is_number, is_boolean (alias is_bool), is_object, is_array, is_null.

LHS paths: dotted segments (response.user.id), bracketed string keys (headers["content-type"]), array indexes in either form (rows[0].name or rows.0.name), and a .length pseudo-segment on arrays and strings. Map-key lookup falls back to case-insensitive matching (useful for headers).

RHS: a double-quoted string (\" and \\ escapes), a number (123, 1.5, -3.2e2), true/false/null (case-insensitive) — or anything else is treated as a path reference, so response.a == response.b and response.id == user_id (scope variable) both work.

Semantics: == is JSON-style loose equality — numeric types compare across int/float; slices/maps compare by JSON stringification. </> comparisons require numerics. contains means substring on strings, element membership on arrays, key presence on objects. jq truthiness: nil, false, "", 0, empty array/map are falsy.

What’s visible to assertions (context root)

Always: duration_ms and every scope variable (non-colliding names at top level). Per protocol:

Protocol	Fields
HTTP	response (JSON-parsed body; raw string fallback), status_code, headers (lower-cased keys), response_size, request_size, ttfb_ms, total_ms
SQL	rows, row_count, rows_affected, columns
gRPC	response, status_code, status_message, metadata, total_ms
script	scope variables + duration_ms only

Templates inside assertions

Assertion strings are Liquid-rendered against the scope before parsing, so response.email == "{{expected_email}}" compares against the resolved value.

Not yet implemented: js: and cel: assertion prefixes parse but currently always fail. Use jq: for complex expressions, or a script step for full logic.

Templating

Interpolation uses Liquid syntax: {{ var }}, {{ var | filter }}, {% ... %} tags. Standard Liquid filters (upcase, downcase, default, json, split, …) are available.

Rendering happens at execute time, not load time — {{token}} sees variables extracted by earlier steps in the same test.

What gets rendered, per protocol:

• HTTP — url, header values, query values, body.payload. Map keys are never templated.
• SQL — connection, query, execute, and string elements of params.
• gRPC — address, service, method, metadata values, and every string inside payload recursively (nested maps/arrays included).
• Assertions — the whole expression string (see above).

variables: blocks themselves are rendered with a two-pass algorithm, so sibling keys can reference each other regardless of declaration order:

variables:
  scratch_ns: "run-{{run_id}}"
  scratch_path: "customer/{{scratch_ns}}/data"   # works, order-independent

Render failures are warnings — the raw value is kept.

Variable scope

Precedence, low → high:

1. Suite chain variables: — root first, deeper suite overrides shallower.
2. Env file (--env or auto-discovered) merged into the root suite.
3. --vars key=value — highest static precedence.
4. before_all mutations — persist in the suite scope; visible to all tests in that suite and to child suites.
5. Test variables: — merged over a snapshot of the suite scope. Test mutations never leak back to the suite.
6. Table row fields — override test variables per iteration (data-driven).
7. before_each mutations, then step extractions as the test runs.

Extracting variables from responses

Declare under the protocol’s response.variables: — extracted values merge into the test scope for subsequent steps and assertions:

http:
  url: "{{baseurl}}/auth"
  method: POST
  response:
    variables:
      token: "response.access_token"        # dotted path
      first_id: "jq: .response.users[0].id" # jq expression

Protocol	Path forms
HTTP	dotted path over response / status_code / headers, or jq:. Dotted paths cannot index arrays — use jq: for that.
SQL	dotted + indexed paths over rows / row_count / rows_affected / columns — rows[0].name works.
gRPC	dotted or jq: over response / status_code / status_message / metadata / duration_ms.
script	test.set_variable(key, value) writes directly to the test scope.

Extraction errors are warnings, not failures — assert on the field too if its presence matters.

Chaining example

tests:
  - name: user-lifecycle
    steps:
      - name: login
        http:
          url: "{{baseurl}}/auth/login"
          method: POST
          body:
            payload: '{"email": "{{email}}", "password": "{{password}}"}'
          response:
            variables:
              token: "response.access_token"
        assertions:
          - status_code == 200
          - response.access_token != null

      - name: create
        http:
          url: "{{baseurl}}/users"
          method: POST
          headers:
            Authorization: "Bearer {{token}}"
          body:
            payload: '{"name": "{{username}}"}'
          response:
            variables:
              user_id: "response.id"
        assertions:
          - status_code == 201

      - name: verify
        http:
          url: "{{baseurl}}/users/{{user_id}}"
          method: GET
          headers:
            Authorization: "Bearer {{token}}"
        assertions:
          - status_code == 200
          - response.id == user_id            # RHS path reference to a scope variable
          - response.name == "{{username}}"

Cookie-based auth needs no explicit chaining at all — each test has one cookie jar shared across its HTTP steps, so a login step’s Set-Cookie is replayed automatically.