Benchmark a script and set a timeout

Goal

A measured distribution for a script, and a timeout chosen from it.

Why bother

The default timeout is 5000 ms. It is a placeholder, and both directions of wrong are expensive.

Without this	With this
A timeout picked as a round number	One derived from the measured tail
Too tight, so it fails under normal load	Set above p99, not above the mean
Too loose, so a hung script holds a worker for five seconds	Bounded near actual behaviour
”Is this script slow?” is an opinion	It is a number

Step 1 — measure

kis script bench transform.js --input '["widget", 2]' --iterations 1000 --warmup 100

  Min:    75.015µs
  Max:    620.271µs
  Mean:   117.263µs
  Median: 102.881µs
  P95:    208.758µs
  P99:    494.018µs

Throughput: 8528 ops/sec

Step 2 — read the right number

Not the mean. The gap between the median (103µs) and the max (620µs) is the whole story: most runs are fast and some are six times slower. A timeout set from the mean fails those.

Statistic	Use it for
Median	What a typical run costs
P95 / P99	What to size a timeout against
Max	The worst the benchmark saw — not a bound, just an observation
Throughput	Capacity planning

Step 3 — act on the number

Take p99, leave headroom for a machine slower than yours, and round up.

Flow
Script

A flow’s protection against a slow step is structural rather than a stopwatch: give the node a retry policy so a transient stall is retried rather than waited out, and route the exhausted case somewhere useful.

- name: transform
  script:
    language: javascript
    file: ./transform.js
  retry:
    max_attempts: 3
    backoff_base: 500ms
    backoff_max: 5s
  error:
    go: give-up

Use the measurement to size the backoff, not a deadline: backoff_base should exceed the p99 you measured, or the first retry starts while the original was still going to succeed.

Where a hard ceiling matters, put it inside the step — timeout on an http: call, a timeout on wait.http — because those are the operations that actually block.

kis script run transform.js --timeout 50

For a script measured at p99 = 494µs, 50 ms is roughly a hundred times the tail — generous enough to survive a loaded host, tight enough that a hung script fails in a twentieth of a second instead of five.

Why warmup exists

The first runs include compilation and cold paths. --warmup runs those and discards them, so the reported figures describe steady state.

Leave it on when sizing for a flow that runs the script repeatedly — a map: node over a thousand items pays compilation once and steady-state a thousand times. Turn it down when you care about first-run cost: a script that runs once per pipeline pays compilation every time, and that cost is real.

Comparing implementations

Benchmarking is also how you settle an argument about two versions:

kis script bench v1.js -i '["widget",2]' -n 2000
kis script bench v2.js -i '["widget",2]' -n 2000

Same input, same iteration count. Compare medians, not means — one outlier moves a mean and tells you nothing.

Verify

Run the benchmark twice. If the numbers move substantially between runs, the machine is noisy and the figures are not yet a basis for a timeout — quieten it or raise the iteration count.