Scripting

Both runners embed a multi-language script engine for logic that doesn’t fit the declarative schema: custom validation, computed test data, side-band checks against files or databases.

v2: script: steps (kis test run)

steps:
  - name: validate-order-totals
    script:
      language: js               # default
      execute: |
        const r = response;      // parsed body of the previous HTTP step
        const bad = r.items.filter(i => i.price <= 0);
        if (bad.length > 0) {
          test.fail("items with non-positive price: " + bad.length, 400);
        }
        test.set_variable("item_count", r.items.length);
        test.success("totals ok", 200);
      timeout: 10s               # default 5s
      params:                    # extra variables merged over the scope
        max_items: 50

Languages

language: accepts js / javascript (goja, default), js:v8 / javascript-v8 / v8, lua, starlark, cel / celgo, expr. (wasm appears in a comment but is rejected.) function: selects an entry point for callable-contract languages (Starlark); it’s ignored for JS.

execute: (inline source) and file: (path relative to the test YAML’s directory) are mutually exclusive.

Injected variables

Scripts see the entire test scope as top-level variables, plus params. When a previous HTTP step exists in the test, they also get:

Variable	Contents
status_code	HTTP status of the last HTTP step
response_body	raw body string
response	JSON-parsed body
headers	response headers, lower-cased keys
duration_ms	last step duration

The test.* API

Function	Description
test.success(msg, code?)	mark the step passed (code defaults 200)
test.fail(msg, code?)	mark the step failed (code defaults 400)
test.skip(reason)	record a skip (reported as 200 + message)
test.set_variable(k, v)	write to the test scope: visible to later steps/assertions
test.get_variable(k)	read from scope
test.log(args...)	write to the step’s log, printed under the step in the console and in the JUnit system-out
test.run_id() / test.test_name() / test.step_name() / test.suite_name() / test.suite_path() / test.source_file()	identity/context accessors
test.now_unix_ms() / test.now_iso8601()	clocks
test.uuid() / test.ulid()	id generators
test.random_int(lo, hi) / test.random_string(n)	randomness
test.env(name)	read an OS environment variable
test.fixture(relpath)	read a file relative to the test’s directory
test.sleep(ms)	sleep
test.set_suite_var(k, v) / test.get_suite_var(k)	publish to / read the suite store: outlives the test, visible to later tests in the suite
test.set_run_var(k, v) / test.get_run_var(k)	same for the run store: visible to every suite in the run

test.set_variable writes to the test’s own scope and is gone when the test ends; the suite/run setters publish deliberately to a wider one. See Assertions and variables for precedence, the declarative export: equivalent, and the ordering caveat under parallel:.

Flat aliases exist for legacy scripts: success, fail, set_variable, get_variable, logwriter, log, print, println, uuid, ulid, sleep.

Other namespaces

Beyond test.*, a script step reaches:

Namespace	Contents
eval	comparison predicates and model-free metrics: eval.equals, eval.json_path_equals, eval.f1, … plus fail-fast eval.must.*
array fs hash json math string template time	general helpers
shell http db git docker kubectl vault secret jq liquid doc imgproc bbox …	the same task implementations the flow runner executes, reached from JS
csv excel parquet entity aggregate compute rest	tabular data: useful for building test data
llm rag embed vector intent guard …	model-backed helpers
cron port osuser	host inspection

The set of namespaces is part of the build, so kis test and test.svc offer the same ones.

Signatures follow the task convention: most take a single map, e.g. csv.read({path: "..."}). shell.execute also accepts a bare string. Each returns a map with success and either the payload or error.

Pass/fail semantics

• test.fail(...) or a non-2xx code passed to success/fail → step failed.
• A runtime error in the script → step errored.
• test.skip(...) → recorded as 200 with the skip message.
• Script finishes without calling anything → 200, passed.
• Timeout (default 5s) fails the step with [script: timed out after X] in the log.

Assertions on a script step evaluate against the scope + duration_ms only: do validation inside the script, or set variables and assert on them.

Legacy: plugin: actions (kis test tests)

steps:
  - name: js-hello
    plugin:
      payload:
        language: js        # js | js:v8 | cel | celgo
        execute: |
          logwriter("hello from the plugin engine");
          success("ok", 200);

The script passes by ending with success(...) and fails by ending with fail(...); a script that ends with neither fails, with that as the reason. The ending can sit inside an if/else or a try/catch, as in the example below.

The data record’s values and the environment are the script’s globals: read username, not {{username}}. Script code runs as written, so a value in the data is always data; code containing {{ or {% is refused with that explanation.

Registered functions

Function	Description
success(message, statuscode)	return success
fail(message, statuscode)	return failure
logwriter(args...)	write to the step’s log (same as test.log)
getdbconnection(url)	open a PostgreSQL connection, returns a connection ID
dbquery(id, query)	run a query → {result: [...rows]}
dbexecute(id, statement)	run a statement that returns no rows → {result: [], rows_affected: n}
closedbconnection(id)	close the connection; any still open close when the step ends
hash.string({data, algorithm})	hash a string → {hash, algorithm}; algorithms include sha256 (default), sha1, sha512, md5, blake2b, xxhash
hash.file({path, algorithm})	hash file contents, same shape
fs.read(path) / fs.glob(pattern)	read a file / read all files matching a glob

DB example:

- name: verify-in-db
  plugin:
    payload:
      language: js
      execute: |
        const id = getdbconnection("postgres://user:pass@localhost:5432/db?sslmode=disable");
        const out = dbquery(id, "SELECT count(*) AS n FROM users");
        closedbconnection(id);
        if (out.result[0].n > 0) { success("rows present", 200); }
        else { fail("no rows", 500); }

Validation expressions

Separate from plugin steps, rest: steps in the hierarchy format run response.validate expressions in the language named by the step’s language: key: js, jq, or celgo.

Security note

Scripts run with real capabilities: filesystem reads and writes, OS env access, shell and ssh execution, docker/kubectl control, and secret/vault reads. A test suite is therefore as privileged as a shell script. Test definitions are code: review them like code, and run collections you trust.