Learning objectives
- Design tenant-safe retrieval for internal incident notes.
- How would you diagnose a RAG answer that misses a known runbook?
- Apply Embedding space to a customer decision and defend the tradeoff.
The mental model
Retrieval-augmented generation gives a model selected reference material before it answers. It can improve access to current or private knowledge, but it does not turn generated text into guaranteed truth. A retrieval system must find the right passages, preserve their source identity, and prevent users from seeing material they are not allowed to access. Embeddings represent content numerically so similarity search can find related items even when wording differs. Similarity is not authority, freshness, or permission. For a customer engineer, the important question is not whether a demo can answer one question, but how it behaves when evidence is missing or access differs.
How it works
Design ingestion and query processing separately. Ingestion extracts text, chooses chunk boundaries, attaches document identity and version, creates embeddings, and updates the index. Chunking should preserve useful context without burying relevant facts in excessive text. Use the same compatible embedding space for stored vectors and queries; changing models or dimensions can require reindexing. Vectorize supplies vector indexing and search capabilities, while the application remains responsible for authorization and source lifecycle.
At query time, authenticate the user and constrain retrieval to permitted material. Do not retrieve everything and rely on the model to hide restricted passages. Validate returned document identities against authoritative permissions where the design requires it. Build context with clear source boundaries and token limits, and ask for an abstention when evidence is insufficient. Evaluate retrieval recall separately from answer quality: an incorrect answer may originate in missing documents, poor chunking, bad ranking, or generation. Deletion must propagate to objects, indexes, caches, and derived records. Record freshness and access-policy assumptions so stale indexed material does not silently outlive its authority.
Create a retrieval debugging record containing the query, embedding model version, applied access filters, returned document identifiers, ranking scores, and source versions. Store sensitive text only when authorized and necessary. A high similarity score should prompt inspection of relevance, not automatic trust. If a source was superseded, test whether the old vector remains searchable. A complete deletion check includes the primary document, derived chunks, vector records, response caches, and any retained review dataset.
Applied example · hypothetical
Hypothetical scenario: an internal assistant answers support engineers from runbooks and customer-specific incident notes. A query about a database error retrieves a highly similar note belonging to another customer. The secure design applies tenant restrictions before source text enters the prompt and validates document permissions. An evaluation case checks that the assistant abstains when only another tenant has the answer. A second case tests a replaced runbook version. The project tracks retrieval quality and access isolation separately because good relevance does not compensate for a privacy failure.
In a customer engagement
Ask who owns the documents, how permissions change, how rapidly updates must appear, and whether deletion includes derived data. Show an example with no authorized evidence, not only a successful answer. Agree on citation behavior and the escalation path when sources conflict. This makes the PoC useful to security and content owners as well as developers.
Interview relevance
A strong RAG interview answer covers ingestion, query processing, authorization, freshness, and evaluation. Explain why a vector database alone is not a complete RAG architecture. Relate agent context assembly to the same source-boundary problem, while distinguishing a proposed Vectorize implementation from any retrieval system you have personally built and verified.
Common mistakes
- Delegating permissions to the prompt
- Ignoring stale indexed copies
- Blaming the model immediately
Key terminology
Embedding space — The numerical representation produced by a particular embedding model in which similarity is measured.
Chunking — Dividing source material into retrieval units while preserving enough context for useful interpretation.
Retrieval recall — The proportion of relevant evidence successfully retrieved for a query or evaluation set.
Explain it without the page
- Design tenant-safe retrieval for internal incident notes.
- How would you diagnose a RAG answer that misses a known runbook?
Further reading
Sources explain the underlying concepts; examples and practice answers are original course material. Recheck current limits and plan eligibility before a real proposal.