Notify Operations

Tenant configuration

Providers are configured per tenant, with credentials as vault references:

notify:
  default_country: "US"
  providers:
    sendgrid:
      apikey: "vault://kis/tenants/{{tenant}}/sendgrid#apikey"

default_country drives country-based SMS routing, the router falls back to it when a recipient’s number does not resolve to a region, which happens more often than test data suggests, because test data is usually all one country.

Caution

Provider credentials are always vault: references, a literal API key here is a key in your git history, and notification providers are a common target precisely because their keys are often the least carefully handled ones in a deployment.

Routing

Per-tenant rules decide which provider serves each recipient:

• Country-based SMS routing: different carriers or aggregators per region
• Primary and fallback chains: when the primary provider fails, the next takes it
• Per-platform push: APNs for Apple, FCM for Android, web push for browsers

Fallback chains are worth configuring before you need them, a provider outage during a product launch is exactly when nobody has time to add one, and the failure is silent from the caller’s side. The send returns, the message never arrives.

The outbox

Async sends land in an outbox and a daemon delivers them. This is what makes async durable: the message survives a restart of the service, the provider, or both.

Outbox depth is the health signal for the whole block. A queue that grows rather than drains means a provider is failing or throttling you, and it will keep growing until someone looks.

Authentication

Routes accept either:

• A service API key, for service-to-service callers, authorised through the platform’s agent and bot configuration
• An end-user JWT, issued by IAM

The in-app inbox routes are user-scoped and require a user JWT, a service key cannot read another user’s inbox, which is the property you want given those routes return message contents.

Suppression

Suppression is enforced at send time, not at list-build time. Opt-outs, hard bounces and complaints all suppress future sends to that recipient on that channel.

Because it applies at send, an opted-out recipient staying on a list is correct rather than a leak, and if they opt back in, sends resume without rebuilding the list.

Adding a provider

A provider is a channel adapter behind a one-method interface plus a descriptor, roughly fifty lines. That is the intended extension point when you need a regional SMS aggregator or an internal channel the platform does not ship.

What to watch

Signal	Why it matters
Outbox depth	A growing queue means a provider is failing or throttling
Per-provider failure rate	Which channel is degraded, before users report it
Suppression hits	Opted-out recipients and bounced addresses
Idempotency collisions	Usually a caller retrying without realising
Fallback activation rate	The primary provider degrading, invisibly to callers

Fallback activation is the one most likely to go unnoticed. Sends succeed, so no alert fires, but you are paying your fallback provider’s rates and inheriting their deliverability.