Core Concepts

The mental model behind AI Flow. Read this once and the API, the authoring guide, and the operations docs will all make sense.

The one-paragraph model

You author a flow (a DAG of tasks). You submit a request to AI Flow over HTTP. AI Flow hands it to the execution engine (the flow engine), which plans the flow and, for each task, asks the orchestrator (the agent fabric) to pick a healthy agent. The agent runs the task and streams the result back. The engine merges results into a shared context, follows the flow’s routing to the next task, and repeats until a terminal node. Everything — the instance, its runs, and the conversational request/response turns — is persisted to a per-tenant PostgreSQL database.

author flow ─▶ submit request ─▶ engine plans ─▶ orchestrator selects agent
    │                                                      │
    └──────────── persisted as instance/runs ◀── agent runs task, streams result

The three layers

Layer	Component	What it owns
Service	AI Flow (aiflow.svc)	HTTP surface, persistence, conversation memory, per-tenant wiring.
Engine	the flow engine	Parse → plan → dispatch → route; control-flow nodes; retry/timeout; pipelines.
Fabric	the agent fabric	gRPC orchestrator↔agent transport; agent health & capacity; agent selection.

AI Flow is intentionally thin — it wires the other two together and remembers everything.

Flow / Workflow / Pipeline

Three words for the same authored artifact: a directed graph of tasks. “Pipeline” is the word the REST layer uses; internally it’s a workflow definition. See [authoring-flows.md](/blocks/ai/AI Flow/authoring-flows/) for how to write one.

A definition is versioned and identified by definitionid. Definitions are delivered per tenant through the forge-meta product-config system (not baked into the binary).

Instance

One execution of a definition. It carries:

• context — the shared bag of data (smap.Map) that tasks read and write,
• states — an ordered history (Created → Executing → … → Completed/Errored),
• metadata — plan, tenant, affinity, and bookkeeping for foreach/map/wait/subworkflow.

Persisted as mlflow_instance.

Run

A single node-execution attempt within an instance, bound to the agent (workerid) that ran it, with its own context/states and resumedata. Control nodes create synthetic run records. Persisted as mlflow_instance_run.

Task & node types

A task is one step. The engine infers the node’s kind from which key you use:

• Task (shell:, print:, httpclient:, …) — dispatched to an agent.
• Assign (assign:/pass:) — inject/reshape context, inline.
• Choice (choice:) — conditional routing, inline.
• ForEach (foreach:) — sequential iteration.
• Map (map:) — parallel fan-out.
• Wait (wait:) — suspend for a signal or timeout.
• SubWorkflow (subworkflow:/flow:) — call a child flow.
• Fail / Succeed — terminate.

Context

The single shared data structure per run. Seeded from vars: + start-time overrides. Written by Assign, task outputs (merged back, with _result as the latest), routing set:, iteration (_item/_index/_results), and signals (_signal). Strings are templated with Liquid ({{var}}); conditions are evaluated by the platform scripting runtime (JavaScript by default).

Conversation & memory

AI Flow adds a chat abstraction on top of pipelines:

• A conversation (mlflow_instance_conversation) is a session bound to an instance.
• Each turn is a request (mlflow_instance_request, holds record.prompt) and a response (mlflow_instance_response, holds the model output).
• Conversation memory = the requests/responses flagged inmemory=true. On each new request, AI Flow replays them as chat history (role:user/role:assistant) into the pipeline’s environment. “Cleaning” memory sets inmemory=false (a soft eviction — nothing is deleted).

Agent & orchestrator

• An agent is a remote worker node. It dials the orchestrator, sends heartbeats with its CPU/memory/GPU/disk metrics and capabilities (tags, loaded models, tenant list, capacity), and executes tasks. Registered in mlflowagentlist.
• The orchestrator is the gRPC server agents connect to. It tracks each agent’s health and capacity and, via the selector, picks an agent for each task.

Selection & selectors

For each Task node the engine asks the orchestrator for a worker. Selectors on the task (tag, model, gpu, freegpu, cpu, memory, id/workerid, group) narrow the choice; otherwise the least-loaded healthy agent wins. Selection is tenant-scoped. If a matching agent exists but is at capacity, the work is queued and retried; if no agent could ever match (bad tag/model), it fails fast.

Affinity & task groups

• Affinity pins tasks to agents across runs (scopes: single-agent, definition, payload). Bindings live in mlflow_affinity_binding.
• Task groups co-locate a set of tasks on one agent within a single run — lighter than affinity, for tasks sharing a workspace.

Overflow queue

When no agent is available, the task is persisted as a mlflow_work_item and the instance goes Queued. A background drain loop retries it (fairly across tenants). Orphaned instances are re-queued on restart. This is what makes bursty load safe.

Multi-tenancy (CPET)

Tenant identity is Customer:Product:Environment:Tenant. It flows from the JWT/header into a per-tenant engine, definition set, and PostgreSQL schema (with row-level security via app.tenant_id). Agents declare which tenants they serve (--tenants, --exclusive).

Suspension & resumption

Flows can pause and resume:

• Wait suspends until a signal (POST /workflow/signal) or a timeout.
• SubWorkflow suspends the parent until the child completes.
• Queued work resumes when an agent frees up.
• Iteration checkpoints let a large ForEach/Map resume mid-collection (opt-in).

Execution types

The definition’s shape picks how it runs:

Type	Chosen by	Behavior
DAG	default	Graph with next:/error: routing.
Array	list: true	Sequential list.
Stateflow	states:	Event-driven state machine.
DataPipeline	pipeline: true	Source/sink streaming (what conversational requests use).

Where things live (quick map)

Concept	Persisted as	Owned by
Definition	forge-meta config → parsed in memory	AI Flow / the flow engine
Instance	mlflow_instance	the flow engine (the platform data layer store)
Run	mlflow_instance_run	the flow engine (the platform data layer store)
Conversation	mlflow_instance_conversation	AI Flow
Request / Response	mlflow_instance_request / _response	AI Flow
Engine record	mlflowengineinstance	AI Flow
Agent	mlflowagentlist	the agent fabric selector store
Queued work	mlflow_work_item	the flow engine
Affinity binding	mlflow_affinity_binding	the flow engine

Next: [authoring-flows.md](/blocks/ai/AI Flow/authoring-flows/) to write one, or [rest-api.md](/api/AI Flow/) to drive it over HTTP.