Data & Templating

Databases, structured data, templating and pipeline variables. Every parameter with its type, default and description, plus worked examples.

DB

Database operations for PostgreSQL, ClickHouse, and DuckDB.

Parameters

Connection

Parameter	Type	Required	Default	Description
driver	string	Yes	-	Driver: postgres, clickhouse, duckdb
host	string	Yes*	-	Database host
database	string	Yes*	-	Database name
username	string	No	-	Database username
password	string	No	-	Database password (vault key or literal)
from_vault	bool	No	true	Whether password is a vault key
path	string	No	-	File path (for DuckDB)
readonly	bool	No	false	Read-only mode (DuckDB)

Operations

Parameter	Type	Required	Default	Description
operation	string	Yes	-	Operation: query, exec, insert, update, delete, bulk
sql	string	No	-	SQL query/statement
queries	list	No	-	Multiple SQL statements
args	list	No	-	Positional query arguments
table	string	No	-	Table name (for insert/update/delete)
columns	list	No	-	Column names
values	list	No	-	Values to insert
rows	list	No	-	Multiple rows for bulk insert
set	map	No	-	Column-value pairs for update
where	string	No	-	WHERE clause
where_args	list	No	-	WHERE clause arguments
returning	list	No	-	Columns to return
on_conflict	string	No	-	ON CONFLICT clause (upsert)
batch_size	int	No	1000	Batch size for bulk operations
setvar	string	No	-	Variable to store result

Examples

Query with Parameters

name: db-query
tasks:
  - name: get-active-users
    db:
      driver: postgres
      host: localhost
      database: myapp
      username: app
      password: db-password-key
      operation: query
      sql: SELECT * FROM users WHERE active = $1 AND role = $2
      args: [true, "admin"]
      setvar: admins

Insert with Returning

name: db-insert
tasks:
  - name: create-user
    db:
      driver: postgres
      host: localhost
      database: myapp
      username: app
      password: db-key
      operation: insert
      table: users
      columns: [name, email, active]
      values: ["John Doe", "[email protected]", true]
      returning: [id]
      setvar: new_user

Bulk Insert

name: db-bulk
tasks:
  - name: insert-events
    db:
      driver: postgres
      host: localhost
      database: myapp
      username: app
      password: db-key
      operation: bulk
      table: events
      columns: [event_type, user_id, data, timestamp]
      rows:
        - [page_view, 1, '{"page": "/home"}', "2024-01-01T10:00:00Z"]
        - [click, 1, '{"element": "button"}', "2024-01-01T10:00:05Z"]
        - [page_view, 2, '{"page": "/about"}', "2024-01-01T10:00:10Z"]
      batch_size: 1000

DuckDB Analytics

name: duckdb-analysis
tasks:
  - name: analyze
    db:
      driver: duckdb
      operation: query
      sql: |
        SELECT category, SUM(amount) as total
        FROM read_csv('/data/sales.csv')
        GROUP BY category
        ORDER BY total DESC
      setvar: category_totals

Namespace: db

Database connection and query functions.

Functions

Function	Description
db.connect()	Establish database connection
db.query()	Execute SELECT query
db.exec()	Execute INSERT/UPDATE/DELETE
db.insert()	Insert row(s) into table
db.update()	Update rows in table
db.delete()	Delete rows from table
db.bulk()	Bulk insert operations
db.close()	Close connection

Connect Parameters

Parameter	Type	Default	Description
driver	string	"postgres"	Driver: postgres, mysql, sqlite, duckdb
host	string	"localhost"	Database host
port	int	5432	Database port
database	string	-	Database name
username	string	-	Username
password	string	-	Password
sslMode	string	"disable"	SSL mode
path	string	-	File path (SQLite/DuckDB)
readOnly	bool	false	Read-only mode

Query/Exec Parameters

Parameter	Type	Description
connectionId	string	Connection ID from connect()
sql	string	SQL query/statement
args	array	Positional arguments
named	object	Named arguments

Insert/Update/Delete Parameters

Parameter	Type	Description
connectionId	string	Connection ID
table	string	Table name
columns	array	Column names
values	array	Values to insert
set	object	Column-value pairs (update)
where	string	WHERE clause
whereArgs	array	WHERE arguments
returning	array	Columns to return

Examples

// Connect to PostgreSQL
let conn = db.connect({
  driver: "postgres",
  host: "localhost",
  database: "myapp",
  username: "app",
  password: "secret"
})

if (conn.success) {
  // Query with parameters
  let users = db.query({
    connectionId: conn.connectionId,
    sql: "SELECT * FROM users WHERE active = $1",
    args: [true]
  })
  log("Found", users.rows.length, "users")

  // Insert with returning
  let newUser = db.insert({
    connectionId: conn.connectionId,
    table: "users",
    columns: ["name", "email"],
    values: ["John", "[email protected]"],
    returning: ["id"]
  })
  log("Created user ID:", newUser.rows[0].id)

  // Update
  db.update({
    connectionId: conn.connectionId,
    table: "users",
    set: { active: true, updated_at: "NOW()" },
    where: "id = $1",
    whereArgs: [newUser.rows[0].id]
  })

  // Bulk insert
  db.bulk({
    connectionId: conn.connectionId,
    table: "events",
    columns: ["type", "user_id", "data"],
    rows: [
      ["login", 1, "{}"],
      ["click", 1, '{"button":"submit"}'],
      ["logout", 1, "{}"]
    ],
    batchSize: 1000
  })

  // Close connection
  db.close({ connectionId: conn.connectionId })
}

// DuckDB for analytics
let duck = db.connect({
  driver: "duckdb",
  path: "./analytics.db"
})

let stats = db.query({
  connectionId: duck.connectionId,
  sql: "SELECT COUNT(*) as total FROM events"
})
log("Total events:", stats.rows[0].total)
db.close({ connectionId: duck.connectionId })

JQ

Evaluate jq expressions against the workflow environment. Uses the gojq library for jq-compatible query evaluation.

Parameters

Parameter	Type	Required	Default	Description
expression	string	Yes*	-	JQ expression to evaluate against the workflow environment
setkey	string	No	-	Variable name to store the result (supports Liquid templates)
queries	string	No	-	Path to a YAML file containing named query definitions
query	string	No	-	Name of a predefined query from the queries file

*Either expression or query (with queries) is required.

Examples

Extract a Value

tasks:
  - name: get-host
    jq:
      expression: ".config_host"
      setkey: db_host

Filter and Count Items

tasks:
  - name: count-active
    jq:
      expression: "[.services[] | select(.active == true)] | length"
      setkey: active_count

Build a New Object

tasks:
  - name: build-summary
    jq:
      expression: "{host: .db_host, port: .db_port, name: .db_name}"
      setkey: db_summary

Dynamic setkey with Template

vars:
  key_prefix: "result"
tasks:
  - name: dynamic-key
    jq:
      expression: ".total"
      setkey: "{{key_prefix}}_total"

Pipeline with jq and Shell

tasks:
  - name: extract-port
    jq:
      expression: ".app_config | fromjson | .port"
      setkey: app_port

  - name: start-app
    shell: |
      echo "Starting app on port {{app_port}}"

Named Query from File

Define reusable queries in a YAML file:

## /etc/kis/queries.yaml
queries:
  get-db-host:
    description: "extract database host"
    expression: ".config.database.host"
    setkey: "db_host"
  count-services:
    description: "count running services"
    expression: ".services | length"
    setkey: "service_count"

Reference named queries in workflows:

tasks:
  - name: get-host
    jq:
      queries: "/etc/kis/queries.yaml"
      query: get-db-host

Named Query with setkey Override

tasks:
  - name: count-svcs
    jq:
      queries: "/etc/kis/queries.yaml"
      query: count-services
      setkey: total_services

Namespace: jq

JQ expression evaluation functions.

Functions

Function	Input	Returns
jq.query()	{expression, data}	{success, value, error}

Examples

// Query data with a jq expression
var result = jq.query({
    expression: ".users | length",
    data: {users: [{name: "alice"}, {name: "bob"}]}
});
console.log("User count: " + result.value);

// Extract nested values
var host = jq.query({
    expression: ".config.database.host",
    data: {config: {database: {host: "localhost", port: 5432}}}
});
console.log("DB host: " + host.value);

// Filter arrays
var active = jq.query({
    expression: '[.[] | select(.status == "active") | .name]',
    data: {items: [{name: "a", status: "active"}, {name: "b", status: "inactive"}]}
});
console.log("Active: " + JSON.stringify(active.value));

Liquid

Liquid template rendering.

Parameters

Parameter	Type	Required	Description
data	string	No	Inline template string
file	string	No	Path to template file
path	string	No	Output file path
src	string	No	Source directory
dest	string	No	Destination directory
setvar	string	No	Variable to store rendered output

Examples

Render Inline

name: liquid-inline
vars:
  name: "World"
tasks:
  - name: render
    liquid:
      data: "Hello, {{name}}!"
      setvar: greeting

Render to File

name: liquid-to-file
vars:
  host: localhost
  port: 8080
tasks:
  - name: render-config
    liquid:
      data: |
        server:
          host: {{host}}
          port: {{port}}
      path: /etc/app/config.yaml

Process Directory

name: liquid-directory
tasks:
  - name: process-all
    liquid:
      src: /templates
      dest: /output

Namespace: liquid

Liquid template rendering functions.

Functions

Function	Description
liquid.render()	Render a template string or file
liquid.process()	Process templates from directory

Render Parameters

Parameter	Type	Default	Description
data	string	-	Inline template string
file	string	-	Path to template file
path	string	-	Base path for includes
vars	object	-	Variables for template

Process Parameters

Parameter	Type	Default	Description
source	string	-	Source directory
destination	string	-	Output directory
include	array	-	Glob patterns to include
exclude	array	-	Glob patterns to exclude
flatten	bool	false	Flatten output structure
vars	object	-	Variables for templates

Examples

// Render inline template
let result = liquid.render({
  data: "Hello, {{name}}! Your score is {{score}}.",
  vars: {
    name: "Alice",
    score: 95
  }
})
log(result.output)  // "Hello, Alice! Your score is 95."

// Render from file
let config = liquid.render({
  file: "/templates/config.yaml.liquid",
  vars: {
    host: "localhost",
    port: 8080,
    env: "production"
  }
})
log(config.output)

// Process directory of templates
let processed = liquid.process({
  source: "/templates",
  destination: "/output",
  include: ["**/*.liquid"],
  exclude: ["**/partials/**"],
  vars: {
    version: "1.0.0",
    buildDate: new Date().toISOString()
  }
})
log("Processed", processed.processed.length, "templates")

Structured

Edit YAML and JSON files programmatically while preserving key order.

Aliases: yaml, json

Parameters

Parameter	Type	Required	Default	Description
path	string	Yes	-	Path to YAML/JSON file to edit
output	string	No	Same as path	Output file path
filetype	string	No	Auto-detect	File type: yaml or json
setvar	string	No	-	Variable to store parsed content
commands	list	No	-	Edit commands to apply

Command Types

Command	Properties	Description
set	key, value, expression	Set a value at key
append	key, value, expression	Append to array or merge into object
delete	key, expression	Delete a key

Expression Targeting

The expression property allows targeting nested maps by matching a key-value pair:

expression:
  key: name      # Find map where this key...
  value: myapp   # ...has this value

Examples

Set Simple Values

name: structured-set
tasks:
  - name: update-config
    structured:
      path: ./config.yaml
      commands:
        - type: set
          key: version
          value: "2.0.0"
        - type: set
          key: debug
          value: false

Set Nested Object

name: structured-nested
vars:
  db_host: "db.example.com"
tasks:
  - name: update-database-config
    structured:
      path: ./config.yaml
      commands:
        - type: set
          key: database
          value:
            host: "{{db_host}}"
            port: 5432
            ssl: true

Target Nested Map with Expression

name: structured-expression
tasks:
  - name: update-service
    structured:
      path: ./docker-compose.yaml
      commands:
        - type: set
          expression:
            key: container_name
            value: api-server
          key: replicas
          value: 3

Append to Array

name: structured-append
tasks:
  - name: add-dependency
    structured:
      path: ./package.yaml
      commands:
        - type: append
          key: dependencies
          value:
            - lodash: "^4.17.21"
            - axios: "^1.0.0"

Delete Key

name: structured-delete
tasks:
  - name: remove-deprecated
    structured:
      path: ./config.yaml
      commands:
        - type: delete
          key: deprecated_feature
        - type: delete
          key: old_setting

Read YAML into Variable

name: structured-read
tasks:
  - name: load-config
    structured:
      path: ./config.yaml
      setvar: config

  - name: show-version
    print:
      message: "Version: {{config.version}}"

Edit JSON File

name: structured-json
tasks:
  - name: update-package
    json:
      path: ./package.json
      commands:
        - type: set
          key: version
          value: "1.2.3"
        - type: set
          key: scripts.build
          value: "tsc && vite build"

SetVars

Set workflow context variables. Supports literal values, file reading, and glob-based file reading.

Parameters

Parameter	Type	Required	Description
setvar	string	No	Variable name (single variable mode)
value	string	No	Literal value (single variable mode)
file	string	No	File path to read into setvar (YAML/JSON auto-parsed)
glob	string	No	Glob pattern — reads matching files into setvar as map
vars	map	No	Map of variable name to literal value
files	map	No	Map of variable name to file path (YAML/JSON auto-parsed)
globs	map	No	Map of variable name to glob pattern (YAML/JSON auto-parsed)

Or directly provide key-value pairs inline (legacy).

Examples

Single Variable

name: setvars-single
tasks:
  - name: set-url
    setvars:
      setvar: api_url
      value: "https://api.example.com"

Read File (Plain Text)

name: setvars-file
tasks:
  - name: read-cert
    setvars:
      setvar: ssl_cert
      file: "/etc/ssl/certs/app.pem"

Read YAML File (Auto-Parsed)

name: setvars-yaml
tasks:
  - name: read-config
    setvars:
      setvar: app_config
      file: "/etc/app/config.yaml"
  - name: use-config
    shell:
      script: echo "DB host is {{app_config.database.host}}"

Read JSON File (Auto-Parsed)

name: setvars-json
tasks:
  - name: read-package
    setvars:
      setvar: pkg
      file: "/app/package.json"
  - name: use-package
    shell:
      script: echo "Package {{pkg.name}} v{{pkg.version}}"

Read Glob

name: setvars-glob
tasks:
  - name: read-configs
    setvars:
      setvar: config_files
      glob: "/etc/app/conf.d/*.conf"

Multiple Variables

name: setvars-multiple
tasks:
  - name: set-config
    setvars:
      vars:
        database_host: "localhost"
        database_port: "5432"

Multiple Files

name: setvars-files
tasks:
  - name: read-certs
    setvars:
      files:
        ssl_cert: "/etc/ssl/certs/app.pem"
        ssl_key: "/etc/ssl/private/app.key"

Multiple Globs

name: setvars-globs
tasks:
  - name: read-config-groups
    setvars:
      globs:
        nginx_configs: "/etc/nginx/conf.d/*.conf"
        app_configs: "/etc/app/conf.d/*.yaml"

Inline (Legacy)

name: setvars-inline
tasks:
  - name: set-values
    setvars:
      username: "admin"
      environment: "production"

Namespace: vars

Workflow variable functions.

Functions

Function	Description
vars.set()	Set workflow context variables
vars.readFile()	Read file content into variables (YAML/JSON auto-parsed)
vars.readGlob()	Read files matching glob pattern into variables

Examples

// Set variables
vars.set({ vars: { api_url: "https://api.example.com", region: "us-east-1" } })

// Read a plain text file (stored as string)
let result = vars.readFile({ path: "/etc/ssl/certs/app.pem" })
// result.context.content = "<file contents as string>"

// Read a YAML file (auto-parsed into object)
vars.readFile({ path: "/etc/app/config.yaml", setvar: "config" })
// result.context.config = { database: { host: "localhost", port: 5432 }, ... }

// Read a JSON file (auto-parsed into object)
vars.readFile({ path: "/app/package.json", setvar: "pkg" })
// result.context.pkg = { name: "myapp", version: "1.0.0", ... }

// Read multiple files (YAML/JSON auto-parsed, others as strings)
vars.readFile({ files: { cert: "/etc/ssl/certs/app.pem", config: "/etc/app/config.yaml" } })

// Read files by glob pattern (YAML/JSON auto-parsed)
vars.readGlob({ pattern: "/etc/app/conf.d/*.yaml" })
// result.context.files = { "/etc/app/conf.d/db.yaml": { host: "...", ... }, ... }

// Read multiple glob patterns
vars.readGlob({ globs: { configs: "/etc/app/*.yaml", templates: "/etc/app/templates/*.tmpl" } })

SetEnv

Set OS environment variables.

Parameters

Any key-value pairs provided are set as environment variables. Values support Liquid template interpolation from workflow vars.

Examples

Set Environment Variables

name: set-environment
vars:
  environment: production
tasks:
  - name: configure-env
    setenv:
      NODE_ENV: "{{environment}}"
      DATABASE_HOST: db.example.com
      LOG_LEVEL: warn

Set Before Shell Task

name: env-before-build
tasks:
  - name: set-build-env
    setenv:
      CI: "true"
      BUILD_NUMBER: "42"

  - name: build
    shell: |
      echo "CI=$CI"
      echo "BUILD=$BUILD_NUMBER"
      npm run build

Print

Output messages to stdout.

Parameters

Parameter	Type	Required	Description
message	string	Yes	Message to print (supports Liquid templating)

Examples

Simple Message

name: print-simple
tasks:
  - name: hello
    print:
      message: "Hello, World!"

With Template Variables

name: print-template
vars:
  name: "John"
  count: 42
tasks:
  - name: show-message
    print:
      message: "Hello, {{name}}! You have {{count}} items."

Multiline Output

name: print-multiline
vars:
  environment: production
  version: "1.0.0"
tasks:
  - name: show-summary
    print:
      message: |
        ========================================
        Deployment Summary
        ========================================
        Environment: {{environment}}
        Version: {{version}}
        Status: Success
        ========================================

Rule

Execute business rules using the Grule rule engine. Supports loading rules from files or inline definitions.

Parameters

Parameter	Type	Required	Default	Description
ruleset	string	No*	-	Inline GRL rules
path	string	No*	-	Single rule file path
paths	list	No*	-	Multiple rule file/directory paths
recursive	bool	No	false	Recursively search directories
facts	list	No	-	JSON files to load as facts
setvar	string	No	-	Variable to store rule output
history	bool	No	false	Include value change history
audit	bool	No	false	Include audit log

*At least one of ruleset, path, or paths is required.

Facts Files Format

Facts files can be specified as:

• filepath.json — Merge at root level
• key=filepath.json — Nest under key

Examples

Inline Rules

name: rule-inline
vars:
  user_age: 25
  user_country: US
tasks:
  - name: check-eligibility
    rule:
      ruleset: |
        rule CheckAge "Check if user is adult" salience 10 {
          when
            Age >= 18
          then
            Result.IsAdult = true;
            Retract("CheckAge");
        }

        rule CheckCountry "Check country eligibility" salience 5 {
          when
            Country == "US" || Country == "CA"
          then
            Result.IsEligible = true;
            Retract("CheckCountry");
        }
      setvar: eligibility

Rules from File

name: rule-file
tasks:
  - name: apply-pricing
    rule:
      path: ./rules/pricing.grl
      setvar: pricing_result

Rules from Directory

name: rule-directory
tasks:
  - name: apply-all-rules
    rule:
      paths:
        - ./rules/validation
        - ./rules/pricing
      recursive: true
      setvar: rule_output
      history: true
      audit: true

Load Additional Facts

name: rule-with-facts
tasks:
  - name: complex-rules
    rule:
      path: ./rules/business.grl
      facts:
        - ./data/config.json
        - pricing=./data/pricing.json
        - limits=./data/limits.json
      setvar: business_result