File fields and the content store

Binary content never lives in a row. A row holds a reference — an opaque string the block stores in a column — and the bytes live in a blob store addressed through the Content block.

Two independent mechanisms bridge to that store, and they do not talk to each other:

Mechanism	Opt-in	Read by	Purpose
type: file field	Field type	The Data API’s own upload/download hooks	Puts bytes in and streams bytes out over /data/rest
metadata.contentstore	Field annotation	content.svc	Routes an entity’s asset field to a named store and folder

A field can carry one, the other, or both. Nothing in the loader ties them together: the field-type path never reads metadata.contentstore, and the annotation lookup never checks the field’s type.

File-typed fields

file is one of the field types in Fields; it compiles to a text column — the per-dialect column types are on Types.

entities:
  - name: claim
    fields:
      - name: reference
        type: string
        modifiers: [required]
      - name: scan
        type: file
        nullable: true

Two hooks act on file fields, both at priority 260, and both installed only when the host process registered the matching callback at boot:

Hook	Phase	Operations	Effect
Write	after validate	create, update	Passes the payload’s file bytes to the storage callback and replaces the field value with the returned reference
Read	after scan	read	Passes each stored reference to the resolver callback and writes the result to a new key, <field>_url

The write hook accepts three payload shapes for a file field and ignores everything else: a map carrying name and data, raw bytes (filename synthesised as <entity>_<field>_<unix-nanos>), and a plain string — which is treated as an already-stored reference and passed through untouched.

The read hook never overwrites the stored reference. scan keeps the raw reference and scan_url is added alongside it. A resolver failure does not fail the query: the message is attached to the response metadata under file:error:<field> and the row is returned without the URL key.

Uploading: multipart create

POST /data/rest/{entity} accepts multipart/form-data in place of a JSON body. Dispatch is on the Content-Type prefix.

Part kind	Target	Rule
Text	Any field	Value is JSON-sniffed when it starts with { [ " - t f n or a digit, otherwise kept as a string
Text	Undeclared field	Accepted and passed to the engine; the validation hook decides
File	file-typed field	Read into {name, content_type, size, data} for the write hook
File	Undeclared field	400 — the whole request fails
File	Non-file-typed field	400 — the whole request fails
File	Two or more parts on one field	400 — single file per field only

The multipart parse holds 32 MiB in memory and spills the remainder to a temporary file. That cap plus the server read timeout are the only size limits: the write hook is built without a per-field maximum size and without a MIME allow-list, so neither is enforced.

Caution

There is no multipart update path. PUT and PATCH take JSON only, so replacing a file means creating a new row or writing the reference string yourself.

Downloading: a flag on the read route

Download is not a separate path. It is download=true on the single-row read, with field naming the column:

GET /data/rest/claim/id/01J8ZC...?download=true&field=scan

The handler loads only the named field and id, extracts the reference, and streams the body from the downloader callback. Response headers: Content-Type (application/octet-stream when the downloader reports none), Content-Disposition: attachment; filename="…" when a filename is known, and Content-Length when the size is known. Streaming is chunked, so response memory does not track file size.

Condition	Status	Code
No downloader registered	501	v2.file_downloader_unconfigured
download=true on a list path (no id)	400	v2.download_needs_id
field omitted	400	v2.missing_field
field not on the entity	404	v2.field_not_found
field is not file-typed	400	v2.field_not_file
Row not found	404	v2.row_not_found
Reference column holds a non-string	500	v2.field_unexpected_shape
Reference is empty or null	404	v2.field_empty
Downloader returned an error	500	v2.download_failed

download matches only the literal value true, in any case — not the truthy set the other request flags accept. field does nothing on any other route.

Wiring status in the deployable service

data.svc registers neither the storage callback nor the downloader callback. Both surfaces are mounted and both refuse:

Request	Result
POST /data/rest/{entity} with multipart/form-data	501 v2.file_storage_unconfigured
GET /data/rest/{entity}/id/{id}?download=true	501 v2.file_downloader_unconfigured

The unconfigured check runs before the download request is validated, so a malformed download request returns the 501 rather than the 400 or 404 it would otherwise earn. Multipart create is the other way round: the entity is resolved first, so v2.entity_not_found and v2.engine_unavailable can precede the 501.

With no callbacks registered, neither file hook is installed. A file field therefore behaves as a plain text column: you can write a reference string through the ordinary JSON create and read it back, and no <field>_url key is ever added. Move the bytes through the Content block and store the reference it returns.

Content stores

A content store is not declared in entity YAML. There is no top-level content-store block, and the loader has no key for one. Stores arrive per tenant from the metadata stream, in the product.contentstore.data block of the tenant’s metadata response, and are parsed into a per-tenant catalogue on every refresh.

The three names a product supplies

Each store binds a blob root to two of the product’s own entities. Those three names are the whole naming contract:

Name	Source key	Falls back to
Store name	name	— required; a store without one is skipped
Asset entity	assetentityname	<name>_<extendassetentityname>, and extendassetentityname itself defaults to asset
Folder entity	folderentityname	<name>_<extendfolderentityname>, and extendfolderentityname itself defaults to folder

A store named media with neither entity named explicitly therefore binds to media_asset and media_folder. Those entities must exist in the tenant’s schema like any others — folders are rows in the folder entity, assets are rows in the asset entity, and the bytes sit in the blob root.

Caution

The blob root is not in this block. It is matched by store name against a contentstores: list in the tenant configuration, whose contentroot entry carries the backend — contentroot.type is os (the default), s3 or azureblob — and its path or credentials. Any other value fails the store’s init. When no tenant-config entry matches the store name, the store still loads and silently falls back to an os root at ./<store name> — a directory relative to the service’s working directory. Uploads then land on the container’s local disk.

The rest of a store’s properties

Key	Type	Default	Meaning
type	string	native	native, s3 or azureblob. Gates provisioning only — see below. It does not choose the blob backend; contentroot.type does
active	bool	true	Inactive stores resolve as “not found”
version.enabled	bool	false	Whether the store keeps asset versions
locale	bool	true	Whether assets are locale-scoped
cacheexpiry	duration	1h	Asset cache lifetime
access	map	—	The store’s own access rules, separate from entity access rules

type is read in exactly two places, both of which test for native: root-folder creation and root-database creation. Root folders are created on demand — on first use of a store with no recorded root, the block looks for the folder named / and creates it if absent. A store of any other type keeps an empty root until something else provisions it.

Caution

An unrecognised type does not fail the load and does not fall back to the default — it parses to azureblob. Only native, s3 and azureblob are recognised, so nativ or local produces a store that is never native and therefore never gets a root folder or a root database, with no diagnostic. An unparsable cacheexpiry behaves differently: that store is dropped from the catalogue entirely and only a log line records it.

Routing a field to a store

A field opts into a store with two annotation keys under metadata::

Key	Type	Meaning
metadata.contentstore	string	Name of the store this field’s content belongs to
metadata.folder	string	Folder path inside that store

fields:
  - name: thumbnail
    type: string
    metadata:
      contentstore: assets
      folder: /thumbnails

The lookup walks the entity’s fields and returns one mapping per annotated field, each carrying the entity name, the field name, the folder path and the resolved store. Requesting a mapping for an entity that is not in the tenant’s schema fails with ENTITY_NOT_FOUND; requesting one with no schema bound fails with SCHEMA_UNAVAILABLE.

data.svc parses the annotation and carries it on the field, but nothing in data.svc reads it, and it appears in no schema export. content.svc runs the mapping lookup against its own compiled copy of the same schema, and uses the result to decide where an entry’s asset field lands.

The catalogue endpoint

GET /data/contentstore/list returns the calling tenant’s catalogue. It is JWT-gated and CPET-scoped. See the HTTP reference for the shared request conventions.

Note

Content serves a route of the same shape at GET /content/contentstore/list, and additionally accepts ?name= to narrow to one store. Each service holds its own copy of the catalogue, refreshed by its own callback, so the two can disagree at any instant.

{
  "contentstore": {
    "assets": {
      "name": "assets",
      "type": "native",
      "assetentityname": "assets_asset",
      "folderentityname": "assets_folder",
      "active": true,
      "version": { "enabled": false },
      "locale": true,
      "cacheexpiry": 3600000000000,
      "parent": "01J8ZC..."
    }
  }
}

parent is the root folder id. The blob root handle, the store’s access rules and its database handle are held in the process and are not serialised. A tenant with no registered stores gets {"contentstore": null}, not an error. A failure to read the catalogue returns 500 v2.contentstore_list_failed.

Caution

The catalogue is process-global and is replaced, not merged, on each metadata refresh: a refresh for one tenant discards every other tenant’s entry until that tenant refreshes in turn. Callers should treat {"contentstore": null} as “not loaded right now” rather than “this tenant has no stores”.

The catalogue comes from the metadata client’s config-change callback, never from the schema definitions themselves. A deployment running with no metadata service — the schemasource: {type: localdir} path in Schema sources — gets no callback, so /data/contentstore/list stays empty there.

Where the two blocks divide

Job	Owner
Declaring file fields and metadata.contentstore annotations	Data API definitions
Holding the per-tenant store catalogue, serving /data/contentstore/list	data.svc
Creating store databases, root folders, assets and folders; reading the field mappings	content.svc
Storing and serving the bytes, transformations, versioning, locales	content.svc

data.svc deliberately does not create content-store databases. It lists what a tenant has and routes nothing.

Related

• Content — asset stores, folders, and the endpoints that move bytes
• Fields — the file type and the metadata block
• Request flags — download and field among the full flag set
• Field protection — classification directives on the columns holding references