Common library
The library every engine except Datapipes binds, and the one a script reaches by default. If you learn one library, learn this one.
Each row below gives the flow task key and the script namespace functions for the same operation. Full parameters are in the atom reference.
Execution
Section titled “Execution”| Operation | Flow task | Script functions |
|---|---|---|
| Shell | shell: | shell.execute, .open, .run, .close |
| Interactive shell | supershell: | supershell.execute |
| Remote shell | ssh: | ssh.connect, .execute, .upload, .download, .close |
| Script | script: | — (you are already in one) |
| Python | python: | — |
| Mojo | mojo: | — |
| Containers | docker:, podman: | docker.build, .push, .pull, .run, .login, .tag, .inspect, .rm |
| Kubernetes | k8s:, kubectl: | k8s.apply, .delete, .get, .exec, .rolloutStatus, .logs, .scale |
shell.open / .run / .close have no task equivalent. They hold a shell session open across
several commands, which is a thing a script can do and a flow node cannot — each node is
independent by design.
- name: build shell: script: make build workingdir: ./src setvar: build_output next: go: report
- name: report print: message: "build said: {{build_output}}"const r = shell.execute({ script: 'make build', workingDir: './src', capture: true,});log.info(`build said: ${r.output}`);setvar and capture: true are the same instruction — hold the output rather than printing it.
Leave both off and the command streams to the terminal instead.
| Operation | Flow task | Script functions |
|---|---|---|
| Files | file: | file.read, .write, .exists, .remove, .copy, .move, .mkdir |
| Pattern matching | glob: | glob.find |
| Archives | archive: | archive.compress, .extract |
| Content extraction | extract: | extract.markdown |
| Structured files | yaml:, json: | — |
| Format-preserving YAML | yamlx: | — |
| JSON query | jq: | jq.query |
| Templating | liquid:, template: | liquid.render, .process |
yaml: and yamlx: are not the same. yaml: reads and writes; yamlx: edits in place while
preserving comments and key order — the one you want when a human also maintains the file.
Transfer
Section titled “Transfer”| Operation | Flow task | Script functions |
|---|---|---|
| HTTP | http:, httpclient: | http.get, .post, .put, .patch, .delete, .request, .download, .upload |
| Secure copy | scp: | scp.upload, .download, .transfer |
| Sync | rsync: | rsync.push, .pull, .sync |
| S3 storage | s3: | s3.upload, .download, .list, .delete |
| Azure Blob | azureblob: | azureblob.upload, .download, .list, .delete |
| Web scraping | scraper: | scrape.web, .pdf, … — pipeline-only |
| Operation | Flow task | Script functions |
|---|---|---|
| Databases | db: | db.connect, .query, .exec, .insert, .update, .delete, .bulk, .close |
| Rules | rule: | — |
The task carries its connection and its statement in one block, and the mode is operation:. The
namespace splits them: db.connect returns a connectionId that every later call takes, and
db.close gives it back. That is the one place where the script form has steps the task does not.
- name: count-orders db: driver: postgres host: "{{db.host}}" database: orders username: "{{db.user}}" password: "{{db.password}}" operation: query sql: "select count(*) as n from orders where status = $1" args: ["open"] setvar: open_orders next: go: report
- name: report print: message: "open orders: {{open_orders}}"const conn = db.connect({ driver: 'postgres', host: dbHost, database: 'orders', username: dbUser, password: dbPassword,});try { const r = db.query({ connectionId: conn.connectionId, sql: 'select count(*) as n from orders where status = $1', args: ['open'], }); return { open: r.rows[0].n };} finally { db.close({ connectionId: conn.connectionId });}Secrets and identity
Section titled “Secrets and identity”| Operation | Flow task | Script functions |
|---|---|---|
| Vault | vault: | vault.get, .exists, .serviceJwt |
| Secret generation | secret: | secret.generate, .encode, .decode, .hash, .validate |
| Encryption | encrypt:, decrypt: | crypt.encrypt, .decrypt, .encryptFile, .decryptFile |
| Hashing | hash: | hash.string, .file, .bytes |
| Certificates | letsencrypt: | letsencrypt.obtain, .renew, .revoke, .info, .list |
The script-side vault namespace reads secrets and issues service tokens. It does not write —
secret creation is a deliberate act with an audit trail, not something a transform should do in
passing.
Source control
Section titled “Source control”| Operation | Flow task | Script functions |
|---|---|---|
| Git | git: | git.clone, .commit, .push, .pull, .status, .addRemote, .removeRemote |
Coordination
Section titled “Coordination”| Operation | Flow task | Script functions |
|---|---|---|
| Distributed lock | lock: | lock.acquire, .refresh, .release, .peek |
| Poll until ready | wait: | wait.http |
| DNS records | dns: | dns.list, .set, .append, .delete, .lookup |
| Identifiers | id: | id.generate |
wait: as a task polls several kinds of condition; the namespace offers HTTP polling only. For
anything else in a script, write the loop — you have one.
Flow-only operations
Section titled “Flow-only operations”These exist as tasks because they act on the run. A script has language constructs that do the same job.
| Task | Does | In a script |
|---|---|---|
print: | Writes into the run log | log.info() |
setenv: | Sets environment variables for later nodes | Set them in the call |
setvars: | Sets run variables | Assign a variable, or vars.set |
suspend: | Pauses the run until resumed | — nothing to pause |
throw: | Fails the node deliberately | throw |
Script-only namespaces
Section titled “Script-only namespaces”Available to a script, with no task equivalent, because they are language services rather than units of work.
| Namespace | For |
|---|---|
log | Log lines |
json | Parse and serialise |
string, array, math | Value helpers |
time, now, tsnano | Clock and timestamps |
uuid, ulid, nanoid | Identifier generation |
template, interpolate | String templating |
config | Service configuration — config.getString, .getBool, .getInt, .getMap, .isSet, .getAppName, .getRootConfig, .getTenantConfig |
vars | Run variables — vars.set, .readFile, .readGlob |
cpet | The tenancy coordinate the script is running under |
Most of these are in the base set that survives --namespaces none. See
What a script can reach.
See also
Section titled “See also”- Atom reference — full parameters for each of these
- Infrastructure library — services, processes, ports
- Tasks and functions — translating between the two