Key takeaways
- Retrieval-augmented generation answers from passages retrieved from your own content, so every answer can point at its sources.
- Permissions have to be enforced at retrieval time, per user. Otherwise the assistant quietly reveals what the person asking could not open.
- Every answer should cite the passages it used, and say plainly when nothing relevant was found.
- An evaluation set of real questions, scored for faithfulness and usefulness, is what tells you whether a change made things better.
- Retrieved text is untrusted input. Documents can carry instructions, so treat prompt injection as a design risk, not an edge case.
How grounded answers work
Retrieval-augmented generation, usually shortened to RAG, splits the work in two. A retrieval step finds the passages in your own content most relevant to the question; a generation step writes an answer using only those passages, and cites them. The model’s general knowledge shapes the language. Your documents supply the facts.
Most quality problems are retrieval problems
- Split for meaning: break documents at headings and sections rather than every few hundred characters, so each passage keeps its context.
- Combine search methods: semantic search finds paraphrases; keyword search finds part numbers, clause references and names. Most good systems use both.
- Keep the metadata: owner, date, version and status travel with each passage, so a superseded policy can be filtered out or flagged.
- Re-index on change: an assistant answering from last month’s handbook is worse than no assistant at all.
Enforce permissions at retrieval time
The assistant must only retrieve what the person asking is allowed to read. Indexing everything into one store and filtering afterwards is how sensitive content ends up in an answer. Carry access rights with each passage, check them on every query against the user’s identity, and log what was retrieved for whom.
Cite everything, and admit the gaps
- Each claim in an answer links to the passage it came from, so the reader can check it in one click.
- When retrieval finds nothing relevant, the assistant says so instead of answering from general knowledge.
- Answers that will drive a decision — a policy exception, a commitment to a customer — go to a person for review before they are used.
Measure it with an evaluation set
Collect real questions with known good answers and the documents that support them. Score every version of the assistant on the same set: did retrieval find the right passages, is the answer faithful to them, and is it actually useful? Without that, a change to how documents are split, or to the model, is a guess.
Treat retrieved text as untrusted
Documents, emails and web pages can contain instructions aimed at the model, such as “ignore your rules and…”. OWASP lists prompt injection first in its Top 10 for large language model applications. Keep the assistant’s permissions narrow, never let retrieved text trigger an action on its own, and log prompts, retrievals and answers so an odd answer can be traced back to its cause.
A build order that works
Start with one question set
One team, one body of content, and a few hundred real questions from the people who will use it.
Get retrieval right first
Measure whether the right passages come back before tuning how answers are written.
Add citations and refusals
Every answer cites its sources. No source, no answer.
Widen carefully
Add content and users only when the evaluation set shows that quality holds.