Verifying Intake

Verifying an Intake deployment end to end: drive every request and hardening path against a scratch directory, then confirm the submissions actually landed. Companion to the forms guide, which covers behaviour.

Before you start

Nothing external to stand up: no database, no gateway, no network dependency. Intake writes to a JSONL write-ahead log and a DuckDB file, both under a data directory you choose — so a scratch directory is a complete environment.

kvm install baas/intake.svc

The smoke test below drives every request and hardening path against a scratch directory, then shuts down cleanly. It touches none of your real data directories.

End-to-end smoke test

Runs the real binary with a scratch data directory and high rate limits, drives every path, then shuts down cleanly. Nothing here touches your real data dirs.

SCRATCH=$(mktemp -d)
intake.svc \
  --http-bind 127.0.0.1:18081 \
  --storage-backend duckdb --wal-jsonl \
  --duckdb-path "$SCRATCH/intake.duckdb" \
  --jsonl-dir  "$SCRATCH/jsonl" \
  --forms-dir  examples/forms \
  --log-output stdout --log-format text \
  --ratelimit-per-ip 1000 --ratelimit-global 5000 &
SVR=$!; sleep 2
B=http://127.0.0.1:18081
IP='-H X-Kis-Client-IP: 203.0.113.7'

sc() { curl -s -o /dev/null -w "%{http_code}" "$@"; }

echo "valid urlencoded : $(sc $IP -d [email protected] -d team_size=25 -d use_case=banking -d consent=true $B/f/demo_request)  (want 200)"
echo "valid json       : $(sc $IP -H 'Content-Type: application/json' -d '{"email":"[email protected]","consent":true}' $B/f/demo_request)  (want 200)"
echo "bad email        : $(sc $IP -H 'Content-Type: application/json' -d '{"email":"nope","consent":true}' $B/f/demo_request)  (want 400)"
echo "unknown form     : $(sc $IP -d [email protected] $B/f/nope)  (want 404)"
echo "honeypot         : $(sc $IP -d [email protected] -d consent=true -d website_url=http://spam $B/f/demo_request)  (want 200, discarded)"
echo "bad content-type : $(sc $IP -H 'Content-Type: text/xml' --data '<x/>' $B/f/demo_request)  (want 415)"
echo "unknown field    : $(sc $IP -H 'Content-Type: application/json' -d '{"email":"[email protected]","consent":true,"evil":"x"}' $B/f/demo_request)  (want 400)"
echo "too large        : $(sc $IP --data "message=$(head -c 70000 /dev/zero | tr '\0' a)" $B/f/demo_request)  (want 413)"
echo "sql-injection val: $(sc $IP -H 'Content-Type: application/json' -d '{"email":"[email protected]","consent":true,"full_name":"x'\''); DROP TABLE demo_request;--"}' $B/f/demo_request)  (want 200, stored literal)"
echo "health           : $(sc $B/health)   ready: $(sc $B/ready)   metrics: $(sc $B/metrics)"

# rate-limit trip (dedicated low limits, fresh IP)
kill -TERM $SVR; wait $SVR 2>/dev/null

To watch the 429 / 503 paths, run a second instance with tiny limits (--ratelimit-per-ip 2 --ratelimit-per-ip-burst 3) and fire ~8 rapid requests from one X-Kis-Client-IP: the first ~3 return 200, the rest 429.

To watch graceful shutdown, send SIGTERM and confirm the log ends with shutdown signal received, draining → Intake stopped (with the counter snapshot) and the process exits cleanly.

Migration checks: point a second boot at the same DuckDB file with a form that adds a field (expect an applied additive migration log), then with a form that retypes an existing field (expect a fatal destructive diff on … startup error).

Verifying the DuckDB file

DuckDB is single-writer, so stop the server first to release the lock, then query the file with the DuckDB CLI:

duckdb "$SCRATCH/intake.duckdb" \
  "SELECT count(*) AS rows FROM demo_request;"

duckdb "$SCRATCH/intake.duckdb" \
  "SELECT email, coalesce(full_name,'') AS full_name FROM demo_request ORDER BY id;"

The second query is the one that matters after the injection case in the smoke sequence: the payload must come back as an inert string in full_name, with both tables still present. If the table is missing, the value was executed rather than stored.

The JSONL WAL is safe to read while the server runs (that’s the point of the WAL) — cat "$SCRATCH"/jsonl/*.jsonl shows one JSON object per accepted submission, honeypot hits excluded.