Authoring Prompts

A product’s prompts are YAML files in its ai/prompts folder. ai-gateway.svc reads the folder through meta.svc for every tenant it is configured with, and watches it: an edited file is served without a restart. A call names a prompt and gives its parameters; the gateway renders the prompt into the request before it goes upstream.

The product’s git history is the prompts’ version history. Each call’s accounting record names the prompt it ran (prompt) and a stamp of the folder it was rendered from (prompt_version), so a record identifies the exact prompt text.

myproduct/
  ai/
    prompts/                     prompt files, at any depth
      simple-prompts.yaml
      few-shot-text.yaml
      entity-few-shot.yaml
      vue-kanban-prompt.yaml
      vue-kis-kanban/            files that samples include
        Example01.vue
        Example02.vue
    models/                      model aliases: see Configuration

Prompt files

• Only .yaml files are prompt files. A .yml file is skipped, and the gateway logs a warning naming it.
• Files may sit in folders of any depth. Folder and file names carry no meaning: a prompt is known by its name.
• A file is a map with a prompts: list, or a bare list of prompts, in one YAML document.
• A file that a sample includes with file: is a sample, not a prompt file, even when it is a .yaml file. Any other file that no sample includes is logged as unused.
• Prompt names are unique across the product’s ai/prompts.
• Unknown keys and wrong types are errors.

Prompt fields

Field	Type	Meaning
name	string, required	The name a call uses
type	zero-shot (default) or few-shot	A few-shot prompt needs samples; a zero-shot prompt may not have samples, samples-prefix or samples-suffix
system-prompt	string	Liquid template. Replaces the request’s system prompt
user-prompt	string, or list of strings	Liquid template; a list is joined with newlines. Sent as a user message
samples-prefix	string	Plain text before the samples
samples-suffix	string	Plain text after the samples
samples	list	Few-shot examples (below)
content-type	string	Fence language of the sample responses when backticks is true
backticks	bool, default false	Wrap each sample response in a fenced block: ```<content-type>
description	string	Shown by the admin API; no effect on rendering
tags	list of strings	Shown by the admin API; no effect on rendering

Sample fields

Field	Type	Meaning
prompt	string, required	The example’s user text
response	string	The example’s answer
file	path	A file whose content is the answer, as is
content-type	string	Overrides the prompt’s content-type for this sample
backticks	bool	Overrides the prompt’s backticks for this sample

A sample has exactly one of response or file. file is relative to the prompt file’s folder and must stay inside ai/prompts: an absolute path, or a path that leaves the folder, is an error.

Templates and parameters

system-prompt and user-prompt are Liquid templates. Every other field, samples and their framing included, is plain text.

Variables are strict. A variable a template uses that the call does not give is refused with 400 (ai-gateway-prompt-parameter-missing), and the message names the variable. It is never rendered as an empty string. Parameters keep their JSON types, so a number prints as written and a list can drive a {% for %} loop.

Simple prompts

Each prompt has a name and the text sent to the model. The {{ parameter }} notation marks the parts a call fills in.

Example: ai/prompts/simple-prompts.yaml

prompts:
- name: email-template-gen
  system-prompt: You are a helpful executive assistant.
  user-prompt: |
    Write {{ templatecount }} email templates to introduce {{ your-brand }} in {{ charactercount }} characters or less
    by weaving in the specific trigger to {{ topic }} on {{ brand }}. Customer's firstname is {{ firstname }}
    and lastname is {{ lastname }}. My signature is {{ signature }}.
- name: linkedin-post-gen
  user-prompt: |
    Suggest titles of {{ count }} LinkedIn posts to introduce {{ product }} in sequence one for each day. The
    features to be highlighted are {{ featurelist }}.

A call names email-template-gen and gives its parameters:

{
  "model": "chat.default",
  "prompt": "email-template-gen",
  "parameters": {
    "templatecount": 5,
    "your-brand": "kis.ai",
    "charactercount": 150,
    "topic": "hotel management software",
    "brand": "Sun and Sands Resorts",
    "firstname": "Joe",
    "lastname": "Pescano",
    "signature": "Wishing a sunny day, AP"
  }
}

The gateway sends two messages upstream: the system prompt You are a helpful executive assistant., then this user message:

Write 5 email templates to introduce kis.ai in 150 characters or less
by weaving in the specific trigger to hotel management software on Sun and Sands Resorts. Customer's firstname is Joe
and lastname is Pescano. My signature is Wishing a sunny day, AP.

linkedin-post-gen has no system-prompt, so a call to it keeps the request’s own system prompt, if it sends one.

Calling a prompt

POST /v1/chat/completions and POST /v1/messages take two fields beside the dialect’s own, and so do POST /v1/messages/count_tokens (which counts the rendered prompt) and each line of a batch file:

Field	Type	Meaning
prompt	string	The name of a prompt the tenant sees
parameters	JSON object	The template variables. Values keep their JSON types

model is still required: it names the alias the call runs on. messages may be left out when the prompt renders a user message: its own user-prompt, or the user-prompt parameter.

POST /v1/chat/completions HTTP/1.1
Authorization: Bearer <token>
X-Customer:       acme
X-Env:            prod
X-Product:        marketing
X-Tenant:         main
X-Kis-Block:      campaigns
X-Kis-Purpose:    email-drafts
X-Kis-Request-ID: 01JB7CY1Q6R0GK09RZAVDQGF11
Content-Type:     application/json

{
  "model": "chat.default",
  "prompt": "email-template-gen",
  "parameters": { "templatecount": 5, "your-brand": "kis.ai", "charactercount": 150,
                  "topic": "hotel management software", "brand": "Sun and Sands Resorts",
                  "firstname": "Joe", "lastname": "Pescano", "signature": "Wishing a sunny day, AP" }
}

The Anthropic dialect takes the same fields:

{
  "model": "chat.default",
  "max_tokens": 1024,
  "prompt": "email-template-gen",
  "parameters": { "templatecount": 2, "your-brand": "kis.ai", "charactercount": 90,
                  "topic": "hotel management software", "brand": "Sun and Sands Resorts",
                  "firstname": "Joe", "lastname": "Pescano", "signature": "AP" }
}

A default prompt on an alias. An alias may name a prompt with spec.prompt.registry_ref. A call to that alias that names no prompt runs the alias’s; a prompt in the body wins over it.

Parameters in a header. X-Kis-Prompt-Vars carries string parameters as a URL-encoded query string (team=finance&tone=formal). When a call sends both, the body’s parameters win key by key.

Which prompt a name finds. The platform’s prompts first, then the tenant’s product’s. Platform names are reserved: a product prompt that takes one is not served (the platform’s is), and every load of that product logs a warning naming the file. Another tenant’s prompts are never visible: two products may each have a prompt of the same name, and each tenant runs its own.

How a prompt is rendered

For a call that runs a prompt, named in the body or by its alias, the request sent upstream is built in this order:

1. System prompt. The rendered system-prompt replaces the request’s system prompt (an OpenAI system message, or the Anthropic system field). A prompt with no system-prompt takes the system-prompt parameter, as given, when the call gives one; otherwise the request’s system prompt stays.
2. Samples. A few-shot prompt’s samples go first, as alternating user and assistant messages: each sample’s prompt, then its response, fenced as ```<content-type> when backticks is true. samples-prefix and samples-suffix are then appended to the system prompt, each after a blank line. A prompt whose template places the samples itself sends none as messages (see Placing the samples yourself).
3. User message. The rendered user-prompt, as a user message. A prompt with no user-prompt takes the user-prompt parameter, as given, when the call gives one.
4. The request’s messages. The request’s own messages follow, in order.

A prompt with only a system-prompt is a system prompt for a conversation the request’s messages carry. A prompt with a user-prompt is a template call, and the request usually carries no messages. A call whose prompt renders no user text and which sends no messages is refused with 400 (ai-gateway-prompt-invalid).

Placing the samples yourself

A few-shot prompt whose template uses {{ samples }} places the samples itself, and they are not sent as messages. {{ samples }} is their text form: samples-prefix, then each sample’s prompt and response on their own lines, then samples-suffix, separated by blank lines.

prompts:
- name: ticket-triage
  type: few-shot
  system-prompt: |
    You classify support tickets into one queue.

    {{ samples }}
  user-prompt: "Ticket: {{ ticket }}"
  samples-prefix: Examples
  samples-suffix: Answer with the queue name only.
  samples:
  - prompt: "My invoice shows the wrong amount"
    response: billing
  - prompt: "I cannot sign in"
    response: access

A call with "parameters": {"ticket": "Refund not received"} sends this system prompt, then the user message Ticket: Refund not received:

You classify support tickets into one queue.

Examples

My invoice shows the wrong amount
billing

I cannot sign in
access

Answer with the queue name only.

Few-shot prompts

A few-shot prompt gives the model a small number of worked examples of the task before the input.

With plain text

Question and answer example

ai/prompts/few-shot-text.yaml

prompts:
- name: few-shot-prompt
  type: few-shot
  system-prompt: You are a helpful assistant, but give precise answers.
  samples-prefix: Give the response as a simple one or two word answer.
  samples-suffix: Do not respond with anything in addition to what is asked. Do not be too verbose.
  samples:
  - prompt: The Eiffel Tower, located in Paris, France, was completed in 1889 and stands 324 meters tall. Where is Eiffel tower located?
    content-type: text
    response: Paris, France.
  - prompt: The Eiffel Tower, located in Paris, France, was completed in 1889 and stands 324 meters tall. How tall is the Eiffel tower?
    response: 324 meters

The prompt has no user-prompt, so the call gives the user’s text as the user-prompt parameter:

{
  "model": "chat.default",
  "prompt": "few-shot-prompt",
  "parameters": {
    "user-prompt": "In 1873, the three cities of Buda, Óbuda (Old Buda), and Pest were united to form Budapest, which was originally named \"Pest-Buda\". Budapest has a population of about 1.7 million people. What is the population of Budapest?"
  }
}

The gateway sends upstream:

#	Role	Content
1	system	You are a helpful assistant, but give precise answers., a blank line, the samples-prefix, a blank line, the samples-suffix
2	user	The first sample’s prompt
3	assistant	Paris, France.
4	user	The second sample’s prompt
5	assistant	324 meters
6	user	The Budapest text

A response the model may give:

1.7 million

The same structure applies to text generation tasks.

With inline code

Generate YAML with inline samples

ai/prompts/entity-few-shot.yaml

prompts:
- name: few-shot-entity
  type: few-shot
  system-prompt: You are an helpful developer, helping a developer create a valid yaml.
  samples-prefix: Give a valid yaml as output.
  samples-suffix: Do not respond with anything in addition YAML. Include the YAML in triple backticks and also add the type yaml when after starting the triple backticks.
  backticks: true
  content-type: yaml
  samples:
  - prompt: Create entity person with fields id, firstname, lastname, age and address
    response: |
      entities:
      - name: person
        fields:
          - name: id
            type: int
            validations: []
          - name: firstname
            type: string
            validations:
              - type: length
                max: 12
          - name: lastname
            type: string
            validations:
              - type: length
                max: 12
          - name: age
            type: int
            validations: []
          - name: address
            type: string
            validations:
              - type: length
                max: 255
  - prompt: Create entity car with fields id, name, brand, type, price
    response: |
      entities:
        - name: car
          fields:
            - name: id
              type: int
              validations:
                - type: required
                - type: unique
                  message: should me unique for all entries
                - type: final
                  message: field cannot be updated
            - name: name
              type: string
              validations: []
            - name: brand
              type: string
              validations: []
            - name: type
              type: string
              validations: []
            - name: price
              type: string
              validations: []

With backticks: true and content-type: yaml, each sample response is sent fenced as a block that opens with ```yaml. This prompt has no user-prompt, so the call can carry the user’s text in messages instead of a parameter:

{
  "model": "chat.default",
  "prompt": "few-shot-entity",
  "messages": [
    { "role": "user", "content": "create entity address with fields id, building, street, locality, city, state, country, code" }
  ]
}

A YAML response the model may give:

entities:
  - name: address
    fields:
      - name: id
        type: int
        validations:
          - type: required
          - type: unique
            message: should me unique for all entries
          - type: final
            message: field cannot be updated
      - name: building
        type: string
        validations: []
      - name: street
        type: string
        validations: []
      - name: locality
        type: string
        validations: []
      - name: city
        type: string
        validations: []
      - name: state
        type: string
        validations: []
      - name: country
        type: string
        validations: []
      - name: code
        type: string
        validations: []

With file inclusion

Generate Vue code

This prompt generates Vue code from samples whose responses are code files kept beside the prompt. The file is a bare list of prompts. Each file: is relative to the prompt file’s folder, so vue-kis-kanban/Example01.vue is ai/prompts/vue-kis-kanban/Example01.vue; its content is the sample’s response, sent fenced as a ```vue block.

ai/prompts/vue-kanban-prompt.yaml

- name: few-shot-kanbam
  type: few-shot
  system-prompt: You are an helpful developer, helping a developer create valid vue code.
  samples-prefix: Give a valid Vuejs code as output.
  samples-suffix: Do not respond with anything in addition to Vue code.
  backticks: true
  content-type: vue
  samples:
  - prompt: Create a kanban board for bug entity with states new, inprogress, fixed, closed. The attribute values are access-key:aid, discriminator:state, date-key:date
    file: vue-kis-kanban/Example01.vue
  - prompt: Another example, a kanban board for a bug-detection entity, having 3 states detected, danger_level, fixed would be like
    file: vue-kis-kanban/Example02.vue

The call:

{
  "model": "chat.default",
  "prompt": "few-shot-kanbam",
  "parameters": {
    "user-prompt": "Create a kanban board for task entity with states new, inprogress, fixed, closed. The attribute values are access-key:id, discriminator:state, date-key:duedate"
  }
}

A Vue response the model may give:

<template>
  <div>
    <KisKanbanViewer ref="kanban" :width=1400 :height=800 access-key="id" discriminator="state" date-key="duedate"
      :buckets="buckets" :data="data" />
  </div>
</template>
<script setup>
import { useEntityDataSource } from '@kis.ai/vue-data'

const { loading, loaded, data, fetchData, previousPage, nextPage, gotoPage } = useEntityDataSource('task', null, { immediate: true, pagesize: 1000 })

const buckets = [
  {
    title: 'New',
    value: 'new',
    style: {
      headerStyle: 'bg-blue-100 text-black items-center p-3',
      bucketStyle: 'bg-blue-100 px-4 rounded-lg'
    }
  },
  {
    title: 'In Progress',
    value: 'inprogress',
    style: {
      headerStyle: 'bg-blue-100 text-black p-3',
      bucketStyle: 'bg-blue-100 px-4 rounded-lg'
    }
  },
  {
    title: 'Fixed',
    value: 'fixed',
    style: {
      headerStyle: 'bg-blue-100 text-black p-3',
      bucketStyle: 'bg-blue-100 px-4 rounded-lg'
    }
  },
  {
    title: 'Closed',
    value: 'closed',
    style: {
      headerStyle: 'bg-green-100 text-black p-3',
      bucketStyle: 'bg-green-100 px-4 rounded-lg'
    }
  }
]
</script>

Loading and reloading

The gateway reads each configured tenant’s ai/prompts, together with its ai/models, through the tenant’s meta source, and validates every file before it serves any of them. A change to a file under either folder loads both again, and the tenant’s prompts and aliases are swapped in whole.

A load that fails changes nothing. A file that does not parse or validate is refused: the gateway logs the file and the problem, and the tenant keeps serving the prompts and aliases it had. These fail a load, each naming its file (and the prompt, where there is one):

• an unknown key, a wrong type, or a prompt file that is empty, holds more than one YAML document, or is a map without a prompts: list;
• a prompt with no name, or a name another file in the product already declares;
• a few-shot prompt without samples, or a zero-shot prompt with samples, samples-prefix, samples-suffix or {{ samples }};
• a sample with no prompt, with both response and file, or with neither;
• a sample file that is absolute, outside ai/prompts, or not there;
• a template that does not parse.

Until a tenant’s first load succeeds, its calls are answered 503 with ai-gateway-tenant-not-loaded and a Retry-After header.

Platform prompts

The operator’s prompt directory (ai-gateway.prompts.path in the service configuration) holds the platform’s prompts, in the same format, re-read on a cycle. Their names are reserved in every tenant’s view (see Calling a prompt). See Configuration.

Inspecting a prompt

GET /admin/v1/prompts/{name} returns the prompt the request’s tenant sees under that name: its name, owner (kis-platform, or the tenant’s key), source file, version (the folder stamp), type, description, tags, system_prompt and user_prompt template text, and the number of samples. It needs the tenant’s administrator; see API.

Errors

Status	X-Kis-Error-Code	When
404	ai-gateway-prompt-not-found	The name is not a prompt the tenant sees
400	ai-gateway-prompt-parameter-missing	A template uses a variable the call did not give; the message names it
400	ai-gateway-prompt-render-failed	A template failed at render time for another reason
400	ai-gateway-prompt-invalid	prompt is not a name, parameters is not a JSON object, or the prompt renders no user text and the call sends no messages

Errors keep the dialect’s envelope; the code is also the OpenAI envelope’s code.