Operations

Reindexing

Reindexing is the operation to plan for, because two ordinary changes force it:

• Changing the embedding model. Vectors from different models are not comparable, so a partially-migrated collection returns nonsense rankings rather than degraded ones.
• Adding an embedded field. Existing documents have no vector for it.

Both mean re-embedding every document, which costs real money at corpus scale. Index into a new collection and switch reads over when it completes, rather than reindexing in place — in-place leaves the collection in the mixed state above for the duration.

Adding a searchable (lexical) field is cheaper and does not require re-embedding.

Embedding cost

Embedding calls go through the AI Gateway, so a bulk index shows up in your gateway accounting attributed to this block. Two consequences worth planning around:

Set a budget on the embedding alias. A bulk reindex of a large corpus can consume a tenant’s daily allowance in one run, and the failure mode is a half-indexed collection.

Use background priority for bulk work. Interactive search-time embedding (the query vector) should not queue behind a backfill; X-Kis-Priority: background on the indexing path keeps a reindex from degrading live search.

Retrieval quality

The metric that matters is whether the right document appears in the top results, and it is not visible from service health.

Signal	Reading it
Zero-result rate	Queries finding nothing — usually vocabulary mismatch
Click-through position	Where users actually find the answer
Queries with results but no engagement	Retrieval is running but ranking badly
p99 search latency by mode	Semantic and hybrid are the expensive paths
Index lag	Time between writing a document and it being findable

Zero-result rate is the one to instrument first. A collection returning nothing for 15% of queries is failing loudly to users and silently to monitoring, and the usual cause is a corpus that does not use the words your users do — which lexical search cannot fix but semantic retrieval often can.

A rise in “results but no engagement” after an embedding-model change is the signature of a reindex that did not fully complete.

Failure modes

A backend is unavailable. Queries against collections on that backend fail. Collections on other backends are unaffected, which is an argument for not putting everything on one.

Embedding generation fails during indexing. The document is not indexed. Bulk operations report per-document outcomes rather than failing the batch, so a partial bulk index needs the failures replayed — check the response, do not assume 200 means everything landed.

A gateway budget is exhausted mid-reindex. Embedding calls start returning 429 and the reindex stalls part-way. This is the case the new-collection-then-switch pattern protects you from: the live collection is untouched.

Vector and document stores disagree. A document deleted from the record store but not the index returns results pointing at nothing. Deleting through the API keeps them consistent; reaching into the backend directly does not.

Capacity

Semantic search cost scales with vector count and dimensionality, not document size. A million short documents is a harder retrieval problem than a thousand long ones, even though the latter is more bytes.

When search latency degrades, check vector count before checking hardware — outgrowing pgvector is usually the answer, and the fix is moving the collection to milvus rather than adding memory.

What to watch

• Zero-result rate — the loudest quality signal, and invisible in service metrics
• Index lag — a bot answering from a stale corpus looks like a model problem
• Embedding spend by collection — reindexes are the spike
• p99 latency by mode — hybrid runs two queries and fuses them
• Bulk indexing per-document failure counts — partial success reports as success