Field protection and data classification

compliances: is the per-field declaration of what a value is and how it must be protected. One list carries three kinds of token: read-output treatment, gates on logging/export/indexing, and at-rest treatment — plus four sensitivity classifications that expand into a default bundle of the others.

What data.svc enforces today

This is a compliance surface, so read this table before you rely on a directive.

Directive	Enforced by	Status in the deployed service
noexport (explicit or classification-implied)	Schema-export renderers	Enforced
noindex (explicit, or implied by encryption/tokenization)	Migration index emitter	Enforced
encrypted blocking a WHERE/HAVING condition	Compile-time safety guard	Enforced
mask/redact/encrypted/tokenize/hash/pii/phi/pci/gdpr blocking an aggregate	Compile-time safety guard	Enforced
access.actions.unmask / access.actions.decrypt guards	Access hook	Enforced
Classification resolution and introspection	Schema resolver, script namespaces	Enforced
pii/phi/pci/gdpr selecting the columns the retention anonymise pass overwrites	Retention runner	Enforced
pii/phi/pci/gdpr listed by the retention admin endpoint	Admin route	Enforced
mask, redact, hidden on read	Read-side compliance hook	Parses; hook not registered
encrypted, tokenize, hash at rest	Write/read encryption hooks	Parses; hooks not registered
nolog	—	Parses; surfaces in introspection only
retention as a field directive	—	Parses; nothing reads it

Read-side masking and at-rest encryption are not wired

The hooks that mask, redact, hide, encrypt, tokenize and hash a value exist in the engine but are not registered in the hook chain the deployed data.svc builds. A field tagged pii today is removed from schema exports, excluded from indexes, refused as an aggregate target, overwritten by the retention anonymise pass, and reported as PII by the retention endpoint — but its value is returned in cleartext on read and written in cleartext to the column.

A compile-time output filter is configured by the default security profile and never invoked, so it adds nothing. Until the hooks are wired, gate sensitive columns with Access rules — Tier 1 and Tier 2 are enforced on every request — and treat compliances: as classification plus the enforced gates listed above.

Entry shape

Each entry is a map. type: is the directive; the other keys are payload read only by specific directives.

entities:
  - name: patient
    fields:
      - name: ssn
        type: string
        compliances:
          - type: pii
          - type: mask
            value: "XXX-XX-____"

Key	Read by	Meaning
type	all	The directive token.
value	mask, retention	Mask pattern, or retention duration.
key	encrypted	Name handed to the engine’s key provider; empty resolves to default.
algorithm	encrypted	Parsed and stored; the encryption path is AES-256-GCM regardless.
category	—	Parsed and stored; nothing reads it.

compliance: (singular) is accepted as an alias and appends one entry to the same list. Unknown type: values are ignored silently — there is no validation pass over the vocabulary.

The vocabulary

Fifteen tokens. Grouping is descriptive, not syntactic: they all go in the same list.

Token	Group	Effect	Payload
mask	output	Replaces the value on read with a masked representation	value: — the pattern
redact	output	Replaces the value on read with [REDACTED]	—
hidden	output	Removes the field from the row entirely	—
nolog	gate	Marks the value as never-log	—
noexport	gate	Removes the field from every schema export	—
noindex	gate	Suppresses index emission over the field	—
encrypted	at rest	Encrypts the value in the column	key:, algorithm:
tokenize	at rest	Stores an opaque token instead of the value	—
hash	at rest	Stores a SHA-256 digest, one-way	—
retention	lifecycle	Marks a retention intent on the field	value: — duration
searchable	opt-out	Suppresses classification-implied encryption	—
pii	classification	Personal data	—
phi	classification	Protected health data	—
pci	classification	Cardholder data	—
gdpr	classification	GDPR-regulated data	—

Two further tokens are recognised only by the compile-time safety guard, as alternatives to searchable when unblocking a filter over an encrypted column: deterministic_encrypted and blind_index. Neither participates in the classification bundle.

Note

Field-level retention is inert: it resolves into the field’s posture and no runner reads it. Row lifecycle is driven by the entity-level retention: block, a separate mechanism with its own anonymise, archive, archive-delete and purge passes — the anonymise pass is the one place a classification changes row data, and it overwrites exactly the pii/phi/pci/gdpr columns. The legacy entity-level compliance: {retentiondays, retentionaction} block parses and is never read.

The classification bundle

Declaring pii, phi, pci or gdpr fills in the protections you did not spell out. Resolution is per dimension: a classification never overrides a directive you wrote.

Dimension	Default applied	Suppressed by
Read output	mask — pattern last4 when pci is among the classifications and no pattern was given, otherwise the default strategy	any of mask, redact, hidden on the field
At rest	encrypted with key: default	any of encrypted, tokenize, hash on the field — or searchable
Logging	nolog	nothing — always applied
Export	noexport	nothing — always applied
Indexing	noindex, whenever the resolved posture ends up encrypted or tokenized	declaring searchable (which removes the encryption that implied it)

The output and at-rest dimensions are suppressed by any directive in their group, not by the matching one. Writing hidden on a pii field suppresses the default mask; writing hash suppresses the default encryption.

Multiple classifications on one field are additive and all are retained. Only pci changes the default mask pattern.

# Resolves to: masked (default strategy), encrypted with key "default",
#              nolog, noexport, noindex.
- name: date_of_birth
  type: date
  nullable: true
  compliances:
    - type: pii

# Resolves to: masked with "last4", encrypted with key "card", nolog,
#              noexport, noindex. The explicit key wins; pci still sets
#              the mask pattern because no output directive was written.
- name: card_number
  type: string
  compliances:
    - type: pci
    - type: encrypted
      key: card

The searchable opt-out

A classified field is encrypted by default, which makes it noindex and — once the at-rest hooks are wired — unusable in a plaintext filter. searchable is the only way out. It suppresses the default encryption and leaves masking, nolog and noexport in place.

- name: email
  type: string
  compliances:
    - type: pii
    - type: searchable   # keep it queryable for lookups

Caution

The compile-time safety guard refuses a WHERE or HAVING condition on a field carrying an explicit encrypted directive unless the field also declares searchable, deterministic_encrypted or blind_index. Two holes in that check:

• It covers comparison expressions only. ORDER BY and GROUP BY over the same field are identifier-checked and then compiled — sorting or grouping ciphertext is not refused.
• It reads the declared list, not the resolved posture, so a field tagged only pii (encrypted by the default bundle) passes and its filter compiles.

Randomised ciphertext never equals a plaintext argument, so a filter that slips through returns no rows.

Mask patterns

value: on a mask entry is either a named shorthand or a character template. Detection is by exact match against the four names first; anything else containing an X or a _ is a character template, and anything else is a fixed replacement string.

Named shorthands

Pattern	Rule	Example
last4	All but the last four characters become *; a value of four characters or fewer is fully masked	4111111111111234 → ************1234
first1	First character kept, the rest become *; a one-character value is fully masked	Alice → A****
email	First character of the local part kept, rest of the local part masked, domain kept verbatim	[email protected] → a****@example.com
phone	All but the last four characters become *, when the value contains more than four digits; otherwise fully masked	555-123-4567 → ********4567

email falls back to the default strategy when the value has no @. A local part of one character yields a***@domain — the mask length is fixed, not derived.

Character grammar

Any other pattern is walked rune by rune against the value.

Pattern rune	Behaviour
X	Consumes one character of the value, emits *
_	Consumes one character of the value, emits it verbatim
anything else	Emitted as a literal; consumes nothing

Two boundary rules:

• Pattern shorter than the value — the remaining value is emitted verbatim. The pattern names the start of the mask, not the whole layout. Write a pattern at least as long as the longest expected input if you want a hard cut.
• Value shorter than the pattern — leftover X and _ runes are dropped; leftover literals are kept.

Pattern	Value	Result
XXX-XX-____	123456789	***-**-6789
XXX-XXX-____	5551234567	***-***-4567
XXXXX	abcde	*****
_____	hello	hello
___XXX	ref990	ref***
XXX	1234567890	***4567890
XXX-XXX-____	12	**--
XXX@___	alicedomain	***@cedomain

Caution

A pattern containing no X and no _ is not a template — it is used as a fixed replacement string. value: "***" replaces every value with the literal ***, and value: "****{last4}" replaces every value with the literal ****{last4}. To mask positionally, the pattern must contain at least one X or _.

No pattern

With value: omitted — including the mask a classification supplies for a non-pci field — the default strategy applies by value length.

Value length	Result
0	unchanged
1–2	every character replaced with *
3–6	first character kept, rest replaced with *
7 or more	first character kept, last four kept, everything between replaced with *

A non-string value under any mask becomes the literal ***. An empty string is never masked. redact replaces the value with [REDACTED] regardless of type.

Compliance on fields declared inside an embeddable is resolved and applied recursively through the parent field, for both object and array shapes. An embeddable stored as a JSON string is left untouched.

The reveal-action model

Protections split into irreversible and revealable:

Protection	Reversible at read	Reveal action
masked	yes	unmask
encrypted	yes, but transparently — the service holds the key	none of its own
tokenized	yes, via the tokenizer	none of its own
redacted	no	—
hashed	no	—
hidden	no	—

There is exactly one per-user reveal action: unmask. Encryption and tokenization are at-rest controls — they protect the database, disk, backups and DBA from the value, not the value from an authorised reader — so the engine decrypts them transparently and exposes no per-user action for them. A field that is both encrypted and masked exposes only unmask.

Requesting a reveal

Query flag	Request effect	Access rule evaluated
?unmask=true	Asks for unmasked values on a read	access.actions.unmask
?untokenize=true	Asks for detokenized values on a read	access.actions.decrypt

Flag names are matched case-insensitively and accept true, t, yes, y, on and 1. Both are read-only: the guards are evaluated on read operations and ignored on create, update and delete.

The extended guard is ANDed on top of the base read guard — it never replaces it. A caller who passes read but fails unmask gets 403 with action guard denied unmask on "<entity>".

entities:
  - name: patient
    access:
      actions:
        read:   { language: expr, expression: '"clinician" in user.roles' }
        unmask: { language: expr, expression: '"phi-reader" in user.roles' }

Caution

The unmask guard is evaluated only if the entity declares one. An entity with a read rule and no unmask rule accepts ?unmask=true from anyone who passes read. Declare unmask: on every entity that carries a masked or classified field.

Note also the naming split: the request flag is untokenize, the rule slot it evaluates is decrypt.

What a granted unmask covers

Once the guard passes, mask and redact are dropped from the field’s read-side treatment. hidden and nolog are not — a hidden field stays absent from the response even with ?unmask=true.

An impersonated request is a read-only actor by default, and unmask/decrypt are deliberately not classified as mutating actions, so an impersonating operator can still request a reveal.

A granted reveal is not written to the audit stream by the deployed service: the audit sink hook fires on writes only, and the compiler-side sink records access denials. Neither records a successful ?unmask read.

Aggregations

The compile-time safety guard refuses COUNT, SUM, AVG, MIN, MAX and every other aggregate over a field declaring any of encrypted, redact, mask, tokenize, hash, pii, phi, pci or gdpr, unless the principal passes the entity’s access.actions.unmask rule. COUNT(*) has no target field and is always allowed.

safetyguard: aggregation SUM(salary) refused — column carries compliance tags [pii] and principal lacks Unmask permission on entity "employee"

This check fails closed: with no unmask rule declared on the entity, no caller can aggregate over the column at all. The failure surfaces as a 500, not a 403.

Where classification shows up

noexport — declared or implied by any classification — removes the field from every variant of every schema export. See Schema export for the per-format detail and for the GraphQL SDL exception.

noindex — declared, or implied because the resolved posture is encrypted or tokenized — makes the migration planner skip any index that names the field in its columns or its covering columns. The index is dropped from the plan silently; the entity still migrates.

Scripts see the resolved posture. data.schema.field(entity, name) and data.schema.fields(entity) return a compliance object, and the pointcut binding ctx.field carries the same data on three keys:

Introspection key	Contents
classifications	The compound tags as declared — subset of pii, phi, pci, gdpr
primitives	The atomic protections in effect, compounds already exploded: encrypted, tokenized, hashed, masked, redacted, hidden, nolog, noexport, noindex
actions	The reveal actions available — unmask, or empty

On ctx.field the keys are complianceClassifications, compliancePrimitives and complianceActions.

GET /admin/data/retention/entities lists, for each entity with an active retention: block, the field names carrying pii, phi, pci or gdpr in pii_fields[]. The retention runner’s anonymise phase overwrites that same set and stamps anonymized_at. A nullable field is set to NULL; a NOT NULL field gets a type placeholder — [redacted] for string/text, 0 for numerics, false for bool, the Unix epoch for date/time/datetime, the all-zero UUID for uuid/ulid, and NULL for every other type. An entity with no classified field is skipped by the phase entirely.

Spellings that silently do nothing

Caution

The bundled example schemas in the repository use an older dialect that the deployed parser does not read. Copying from them produces a schema that loads with the protection missing and no diagnostic.

Wrong	Right
type: masked	type: mask
pattern: "****{last4}"	value: last4
type: tokenized	type: tokenize
keyref: pii_key	key: pii_key
provider: vault	— (no such key)

Two more that parse but never reach the engine:

• hidden: true as a field key is dropped by the loader. Hiding a field is compliances: [{type: hidden}].
• compliances: declared on a custom type is parsed and stored, but is never merged onto the fields that use that type. Declare protection on the field.

The writeonly modifier — which strips a field from read responses — is applied by the same read-side hook as masking, so it is unwired for the same reason. It does still exclude the field from the output variant of a schema export.

Worked example

entities:
  - name: applicant
    fields:
      - name: id
        type: ulid
      - name: email
        type: string
        validations:
          - type: email
        compliances:
          - type: pii
          - type: searchable          # keep it filterable

      - name: ssn
        type: string
        compliances:
          - type: pii
          - type: mask
            value: "XXX-XX-____"      # overrides the default mask
          - type: encrypted
            key: pii_key

      - name: card_number
        type: string
        compliances:
          - type: pci                 # mask defaults to last4

      - name: internal_score
        type: integer
        compliances:
          - type: hidden              # never returned, never revealable

      - name: consent_ip
        type: string
        compliances:
          - type: gdpr
          - type: hash                # suppresses the default encryption
    access:
      actions:
        read:   { language: expr, expression: '"underwriter" in user.roles' }
        unmask: { language: expr, expression: '"pii-reader" in user.roles' }

Resolved posture, per field:

Field	Primitives	Reveal actions	Indexable
email	masked, nolog, noexport	unmask	yes
ssn	encrypted, masked, nolog, noexport, noindex	unmask	no
card_number	encrypted, masked (last4), nolog, noexport, noindex	unmask	no
internal_score	hidden	none	yes
consent_ip	hashed, masked, nolog, noexport	unmask	yes

ssn carries an explicit encrypted directive and no searchable, so any query with a WHERE or HAVING condition on ssn is refused at compile time; ordering or grouping by it still compiles. email is filterable because searchable removed its encryption. Every field except internal_score is refused as an aggregate target unless the caller passes the unmask rule.

Of that, the deployed service enforces the export removal, the index suppression, the filter and aggregate refusals, and the unmask guard. The masking, encryption and hashing columns of that table describe declared intent, not runtime behaviour.