Core Concepts

The cascade

Every inbound message walks an escalating series of stages. A stage that resolves the message above its confidence threshold stops the cascade; otherwise the message escalates.

Crucially, a cheap stage can resolve both understanding (what the user meant) and response (what to say back). A matched rule carrying a canned answer short-circuits the entire pipeline — no model is invoked at all.

Stage 1 — Rules and regex

Microseconds, zero model cost. Pattern rules declared in YAML, with a scripting escape hatch for matching too awkward to express declaratively. Handles exact patterns, known commands and structured input, and can carry canned responses.

Stage 2 — Dynamic model

Fast and cheap. A lightweight model generated from your stage-1 rule corpus during the build. It generalises past the literal patterns, catching the near-misses the regex does not quite match — the bridge between brittle rules and a real classifier.

Stage 3 — Fast classifier

Milliseconds, tiny cost. An intent classifier trained on your taxonomy plus labelled examples during the build. Catches known intents phrased in unfamiliar ways.

Stage 4 — Small language model

Low cost, via the AI Gateway. Handles moderate complexity, genuine ambiguity and multi-intent messages. Produces intent, entities, and often the response itself.

Stage 5 — LLM

Slow and expensive, via the AI Gateway. A frontier model for anything the cheaper stages cannot handle — novel questions, real reasoning, nuance. Its threshold is 0.0: it is terminal and always resolves.

Note

Human handoff is designed as stage 6 and slots in as a new terminal stage without changing the architecture. Until it ships, stage 5 is the end of the cascade.

Calibrated confidence

This is the subtle part, and worth understanding before you tune thresholds.

The five stages produce “confidence” in fundamentally different ways. A regex match is binary. Classifier softmax scores are famously miscalibrated. An LLM’s self-reported confidence is unreliable. A raw 0.85 from stage 3 and a raw 0.85 from stage 5 do not mean the same thing — and the cascade’s entire cost/quality tradeoff depends on those numbers being comparable.

So each stage gets a calibration function that maps its raw score onto a common probability-of-correctness scale. After calibration, 0.85 from any stage means “85% likely this resolution is actually right”, and a threshold becomes a statement about acceptable accuracy rather than an arbitrary knob.

Calibration functions are fitted during the build, not at runtime: the pipeline runs the cascade against labelled examples, measures each stage’s real accuracy at each confidence level, and bundles the result with the bot artifact.

Rules are the special case. A regex has no native confidence, so a rule’s confidence comes from its historical precision — a rule that has been right 99% of the times it fired scores 0.99; one that frequently fires on messages that turn out to need escalation scores lower. Learned from firing history, not declared by you.

Caution

Calibration drifts as traffic changes. Drift is monitored in production by comparing predicted correctness against actual outcomes, and a drifting stage is flagged for rebuild. Rebuilding is developer-triggered — the platform will tell you the calibration is stale, it will not silently retrain your bot.

Resolution paths

Once the cascade produces an intent, the orchestrator picks a path from your orchestrate.handlers mapping:

Path	What happens	Cost
canned	Fixed response from the rules	None
skill	Invoke a skill — order lookup, account action	Skill-dependent
rag	Retrieve from a knowledge collection, generate a grounded answer	One model call
generate	Open-ended generation using conversation context	One model call
tool	Dispatch a tool over the platform tool protocol	Tool-dependent
flow	Delegate bounded multi-step autonomous work	Flow-dependent
escalate	Trigger escalation	None
script	Custom logic handles the response entirely	Yours

Where the bot stops

There is a real design tension between conversational and task-oriented systems, and plenty of requirements cross it — “the bot should book me a flight”.

The rule this block holds to: AI Bot owns the conversation; AI Flow owns bounded autonomous task execution. When a conversation needs multi-step work — call APIs, reason over results, take actions — the orchestrator routes to a flow via the flow path, hands over the task with conversation context, receives the outcome, and carries on talking.

AI Bot deliberately does not grow agent semantics. It delegates them. That boundary is what keeps the conversational runtime small enough to reason about.

Conversation state and identity

Identity resolves across channels, so the same person on web and on Slack is one user with one history rather than two strangers. Conversations are multi-turn and stateful, with sessions backed by AI Memory for human-facing history.

Streaming

On channels that support it, generative paths (generate, rag, sometimes flow) stream token by token. Non-generative paths (canned, a fast skill) return immediately, with typing indicators where the channel has them. Streams are accumulated into conversation history on completion, so a streamed answer is stored the same as a non-streamed one.

Next

• Configuration — the bot definition in full
• Operations — model generation, analytics and compliance