Remote entities

A remote entity has no table. Every operation against it is turned into exactly one outbound HTTP request, and the response body is mapped back into rows. The engine never compiles SQL for it, so everything the compiler owns — row-level security, field masking, joins, ORDER BY, LIMIT — is absent from the request path.

Declaring one

Set materialization: remote on the entity and add a remote: block.

entities:
  - name: shipments
    materialization: remote
    fields:
      - name: id
        type: string
      - name: reference
        type: string
      - name: status
        type: string
    access:
      actions:
        read:   { language: expr, expression: "true" }   # role-based rules do not
        create: { language: expr, expression: "true" }   # work here — see Identity
    remote:
      name: carrier
      base_url: https://carrier.example.com
      timeout: 5s
      auth:
        type: bearer
        token: ${env.CARRIER_TOKEN}
      getlist:
        pre:
          template:
            url: /v1/shipments
            method: GET
            headers:
              X-Account: "{{user.tenant}}"
        post:
          expression: .data.items
      create:
        pre:
          template:
            url: /v1/shipments
            method: POST
            body: |
              {"reference": "{{payload.data.reference}}"}
        post:
          expression: .data

Key	Type	Default	Meaning
name	string	—	Short identifier for the remote system. Logs and metrics only.
protocol	string	—	Parsed and never read. See the caution below.
base_url (alias baseurl)	string	—	Endpoint root. Relative template URLs are joined onto it.
path	string	—	Fallback URL used when an operation’s template sets no url.
timeout	duration string	10s	Per-request wall clock. Unparseable or non-positive values fall back to the default.
auth	map	—	Credential scheme. See Auth.
create get getlist update delete	map	—	Per-operation request template and response expression.

The block is read key by key. Any other key — including a misspelling of one of these — is ignored without a message, and materialization: itself falls back to table when misspelled.

Caution

protocol: graphql parses but changes nothing: no code reads it. Both values behave identically. To call a GraphQL upstream, declare the query yourself in pre.template.body and POST it.

Operation blocks

Block	Dispatched for	Default method
create	Create	POST
get	Read pinned to a single primary-key value	GET
getlist	Every other read	GET
update	Update, and Restore	PUT
delete	Delete	DELETE

An operation block with neither pre nor post is dropped at load. Attempting an operation whose block is missing fails at request time — a remote entity never falls back to SQL.

Restore maps onto update; there is no separate restore block. The soft-delete rewrite the SQL path performs does not happen here, so ?purge=true reaches nothing and the upstream service decides what a delete means.

URL resolution

1. The operation template’s url, after substitution. Absolute (http://, https://) is used verbatim; anything else is treated as relative.
2. An empty url falls back to remote.path, substituted the same way.
3. A relative result is appended to base_url with its trailing slash trimmed. With no base_url, the relative value is used as-is.

Auth

auth.type	Header sent
bearer	Authorization: Bearer <token>
apikey	<auth.header>: <token>, header defaulting to X-API-Key
basic	Authorization: Basic <base64(user:password)>
anything else, or absent	none

Matching is case-insensitive. An empty credential sends no header at all rather than an empty one. token, user and password expand ${env.NAME} and ${secret.NAME} only — they are substituted with no bindings attached, so a {{…}} reference inside a credential always renders empty. You cannot build a per-tenant token out of {{user.tenant}}.

Template substitution

url, method, body and every header value are substituted before the request is built. A request carrying a non-empty body and no explicit Content-Type header is sent as application/json.

Form	Resolves to
{{payload.data.<field>}}	The mutation payload after the pre-execute hooks have shaped it.
{{payload.id}}	The primary-key value the engine was able to recover.
{{filter.<field>}}	A value from the request’s filter map.
{{user.id}}	The engine’s configured user identity. Empty on data.svc — see Identity.
{{user.tenant}}	The tenant key.
{{user.roles}}	The engine’s configured roles, rendered with Go slice formatting — [a b], and [] when there are none. Never the empty string.
{{entity}}	The entity name.
${env.NAME}	Process environment variable. Not applied to url.
${secret.NAME}	Empty string on data.svc — see the caution below.

Caution

Substitution fails open. A path that resolves to nothing renders as an empty string, so a mistyped {{payload.data.custmer_id}} sends a request with an empty value instead of failing. ${env.X} and ${secret.X} are only expanded in bodies, headers and credentials — a ${…} in url renders empty.

Caution

${secret.NAME} always resolves to an empty string on the deployable service: the secret resolver is a deliberate no-op and nothing overrides it. Use ${env.NAME} and supply the value through the service’s environment, or the auth header is silently omitted. Fetching credentials from Vault is not wired into this path.

Response extraction

The body is read with a hard 32 MiB cap and parsed as JSON. The cap truncates rather than errors, so a larger response is cut mid-document and then decodes to nothing. A body that is not JSON — an empty DELETE response, for example — decodes to nothing rather than erroring too.

post.expression is a jq expression applied to the parsed body. An empty expression returns the parsed body unchanged. The result is then normalised:

Expression yields	Rows returned
An object	One row.
An array	One row per element; a non-object element becomes {"value": <element>}.
A scalar	One row, {"value": <scalar>}.
Nothing, or null	No rows.
Several values	The values as a list, normalised as an array.

Whatever the expression yields is what the caller receives. The entity’s declared fields: do not project, coerce, default or mask the response.

Caution

The bundled example schemas in the repository declare remote blocks with params: under pre.template and expressions of the form response.body.rates. Neither works: params is not a key the loader reads, and response.body.rates is not valid jq — it fails to compile with function not defined: response/0, surfacing as remote: extract: compile post.expression: …. Only url, method, headers and body are read from a template.

The execution path

The engine dispatches on materialization before it does anything else, then runs a reduced version of the hook pipeline around the HTTP call.

Phase	Table entity, read	Remote, read	Table entity, write	Remote, write
BeforeValidate	runs	runs	runs	runs
AfterValidate	runs	skipped	runs	runs
compile to SQL	runs	skipped	runs	skipped
BeforeExec	runs	skipped	runs	skipped
execute	driver	HTTP request	driver	HTTP request
AfterExec	—	—	runs	runs
AfterScan	runs	runs	—	—

Consequences, in the order you meet them:

• Pointcuts registered at BeforeExec never fire on a remote entity. That is BeforeCascade, OnBulkChunk, BeforeBegin, BeforeSCDInsert, BeforeSCDExpire, BeforeIndex, BeforeReindex, BeforeFieldEncrypt and BeforeCacheRead.
• On reads, AfterValidate is skipped as well, so OnAccessDeny, OnRLSFilter and any read-side AfterValidate work do not run.
• Payload edits still stop mattering after AfterValidate, exactly as on the SQL path — the payload is snapshotted into the outbound body at that point.
• The relation eager-load step is part of the SQL read path only, so ?include= resolves nothing on a remote entity.

Declarative machinery that still applies to a remote entity: field defaults, field and entity validations, audit-column stamping, stateflow guards, before_* triggers, Before* pointcuts on create/update/delete, After* pointcuts at AfterExec, the read-side BeforeQuery / BeforeAggregate and AfterQuery / AfterAggregate pointcuts, and read-side computed fields at AfterScan.

Query capability

Reads reach the upstream service through the filter bindings and nothing else. Dispatch happens before the engine builds a query, and an AST the caller already assembled is discarded unread.

Capability	Against a remote entity
Filtering	Only as {{filter.<field>}} bindings you place in the template yourself, and only from a surface that populates them. Operators, AND/OR nesting and negation are not represented — a binding is a bare value.
Sorting	Not forwarded. ?order_by= is ignored.
Pagination	Not forwarded. ?limit=, ?offset= and cursors are ignored; no cursor is minted and no page block is returned.
Projection	Not forwarded. ?select= is ignored.
Relations / joins	Not available. ?include= resolves nothing, and an include of a remote entity from a local parent compiles SQL against a table that does not exist.
Aggregates	Not available. Aggregate clauses are discarded with the rest of the AST; the response is whatever the template returned.
Row cap	The 10,000-row default cap is never applied — dispatch returns before the engine injects it. The upstream response is returned whole, up to the 32 MiB body cap.
Count	count in the response envelope is the number of extracted rows, not an upstream total.

Anything the upstream service should filter, sort or paginate by has to be passed explicitly in the template, from a {{filter.…}} binding or a header.

Which surface you call from decides whether the filter bindings carry anything at all.

Read surface	Filter bindings	Block dispatched
GET /rest/{entity}	empty	getlist
GET /rest/{entity}/id/{id}	empty	getlist
POST /rest/{entity}/search	empty	getlist
GraphQL list query with filter:	the filter argument	get when the filter pins the primary key, else getlist
GraphQL by-id query	id	get, when the entity’s primary key is named id
PATCH /bulk?entity=…&filter=<k>=<v>	the repeated filter pairs	get when one pins the primary key, else getlist
data.query, data.find_one, data.find_by_id from a script	empty	getlist

Caution

The REST read handlers compile the query string into a query and never populate the filter map. So a REST read of a remote entity always dispatches getlist and always renders {{filter.…}} empty — including GET /rest/{entity}/id/{id}, which looks like a single-record read and is not one. The script data.* read functions behave the same way: data.find_by_id puts the id in a WHERE clause on an AST, not in the filter map, so it too dispatches getlist with nothing bound. GraphQL and PATCH /bulk are the only surfaces that populate filters.

Writes fare better, but not uniformly:

Operation	What reaches the template
Create	The whole request body as {{payload.data.*}}; {{payload.id}} only if the body carried one.
Update	{{payload.data.*}} from the body, {{payload.id}} and {{filter.<pk>}} from the path id.
Delete	Nothing. The path id travels in the discarded mutation AST, so {{payload.id}} and {{filter.<pk>}} are both empty.
Restore	Nothing, for the same reason. The restore route sets neither payload nor filters, so the update block fires with every binding empty.

Caution

An HTTP DELETE or restore on a remote entity cannot address a specific record — the template has no binding carrying the id. Model the deletion as an update, whose path id does reach the template.

Security

Control	On a remote entity
Service allow-list (access.services)	Not enforced. Tier 0 is a compiler check and nothing compiles; the runtime hook applies it only when a host supplies a calling-service identifier, and data.svc does not. Declaring services: therefore does not take a remote entity off the air the way it does a table entity.
Per-action rules (access.actions)	Enforced, deny-by-default like everywhere else.
Row-level security on create	Enforced — the create-side rule runs in-process at AfterValidate.
Row-level security on read, update, delete	Not enforced. It is injected as a WHERE predicate by the compiler, which never runs.
Field-level read permission (access.fields)	Not enforced. It is a compile-time projection check, and nothing compiles.
Compliance masking (compliances:)	Not enforced — but not for a remote-specific reason. The read-side masking hook is registered nowhere in data.svc, so it is off on table entities too.

An entity with no access rule for an operation is denied for external callers, so a remote entity needs its access.actions declared like any other. But treat row-level security and field protection as unavailable here: a rule you declare on a remote entity is accepted and, for reads, does nothing.

Identity

The remote path builds its rule context from the engine’s configured identity rather than from the request principal. On data.svc the engine carries the tenant key and no user, which means:

• user.id, user.user_id, user.roles and the flat aliases user_id and roles are empty in access rules on the remote path. A rule such as '"logistics" in user.roles' denies every caller.
• tenant, user.tenant, user.tenant_id, entity, action (alias operation), user.service and user.session_id resolve normally — the last two are read from the request context.
• {{user.id}} and {{user.roles}} render empty in templates. {{user.tenant}} is populated.
• createdby / updatedby are stamped blank when the entity declares them.

Write rules for remote entities against tenant, entity, action or user.service.

Failure and timeouts

Each request is capped by remote.timeout, defaulting to 10 seconds. The cap is wall-clock for the whole exchange, applied by cloning the shared HTTP client per dispatch, so one entity’s timeout never leaks into another’s. There are no retries and no circuit breaker: one operation is one attempt.

Condition	Error
No block for the attempted operation	remote: entity "<name>" has no <op> operation
Block present but no pre.template	remote: entity "<name>" <op> op has no template
Upstream status ≥ 400	remote: upstream <status>: <first 512 bytes of the body>
Timeout, DNS failure, connection refused	remote: HTTP <method> <url>: <cause>
post.expression invalid or failing	remote: extract: <parse|compile|eval> post.expression: <cause>

The engine wraps each of these before the HTTP layer sees it — exec: remote query: … on a read, exec: remote <op>: … on a write.

Caution

None of these map to a gateway status. The REST error mapper classifies by type first and then by substring, and remote errors match neither — so an upstream failure surfaces as 500, not 502. Worse, the classifier lower-cases the whole message, upstream body included, and takes the first match in this order: access denied / not authorised / not authorized / forbidden → 403; not found → 404; validation / invalid / is required → 422. The upstream’s wording decides your API’s status code.

Caution

The error message reaches the caller. It carries the request URL, and for an HTTP error the first 512 bytes of the upstream response body. Assume anything an upstream service puts in an error body is visible to whoever made the API call.

Remote operations never report an affected-row count — the router returns -1 unconditionally, regardless of what the upstream service said. The response envelope’s count is the number of rows the expression produced.

Reaching a remote service from a script

Scripts cannot make outbound HTTP calls. The httpGet, httpPost, httpPut and httpDelete service functions are registered and callable but every one returns servicefn: not yet implemented, and a script written against them runs right up to the failing call. The same is true of cacheGet/cacheSet/cacheDelete/cacheInvalidate, publish, publishBatch, getState, getTransitions, canTransition, getStateHistory, config, configMap, featureFlag, and the five content* functions.

A remote entity is the supported route. The data.* namespace re-enters the engine, so calling data.query or data.create on a remote entity dispatches through the router and runs the same reduced chain, access rules included. Reads from a script always land on getlist with an empty filter map, so a script that needs to address one upstream record has to reach it through a getlist template that carries the discriminator in a header or a hard-coded path. The same applies to a scripted endpoint: it reaches an upstream service by reading or writing a remote entity, not by calling out itself.

Migrations and storage

The migration planner skips every entity whose materialization is not table, so a remote entity produces no DDL and no table exists for it. Nothing in the request path needs one — but the two places that compile SQL without checking materialization do: a relation eager-load into a remote entity, and the engine’s bulk-insert path. Bulk import over HTTP is unaffected, because it creates row by row through the normal mutation path.