Enterprise RAG earns trust by citing every claim and declining when it cannot.

Retrieval-augmented generation answers from your documents instead of memory.

The term comes from a 2020 paper by Patrick Lewis and colleagues, which paired a language model with a searchable index of documents. That paper noted that providing provenance for a model's answers was still an open problem. For a company, provenance decides whether the system can be trusted at all.

A working pipeline has more steps than the diagram usually shows:

1. Parse. Turn PDFs, Word files, spreadsheets, scans and emails into clean text, and keep the page numbers. Scans and Arabic documents need OCR, with its read quality checked on samples.
2. Split. Cut documents into passages small enough to retrieve and large enough to make sense on their own.
3. Index. Store each passage for keyword search and for meaning-based search, with the permissions of its source file attached.
4. Retrieve. Find the passages that bear on the question, for this user only.
5. Rerank. Put the strongest evidence first.
6. Answer. Write the answer from those passages only, with a citation on every claim, or decline.

Retrieval reduces made-up answers without removing them.

In a preregistered study published in the Journal of Empirical Legal Studies in 2025, researchers at Stanford and Yale found that leading AI legal research tools built on retrieval produced hallucinated answers between 17% and 33% of the time. That is fewer than a general-purpose chatbot, and far from the hallucination-free research claimed for those tools.

NIST's Generative AI Profile names the problem confabulation: the production of confidently stated but erroneous or false content. OWASP lists misinformation among the ten main risks of LLM applications. Neither treats it as solved. The design question is therefore how to make every answer checkable, and how to make the system decline when it cannot check.

A bigger context window leaves this unchanged. Research by Nelson Liu and colleagues found that models use information at the start and end of a long input best, and that performance degrades when the relevant passage sits in the middle. Finding the right three passages still beats pasting every document into one prompt.

Retrieval decides the answer.

What the model reads bounds what it can say, and that puts retrieval ahead of model choice. Two techniques have public evidence behind them. Hybrid search, keyword search (BM25) combined with meaning-based vector search, catches exact terms such as a contract number or a product code that vector search alone misses. Reranking the candidates before the model sees them puts the best evidence first.

Anthropic published results in September 2024 on its own test sets. Adding context to each passage before embedding cut failed retrievals among the top 20 passages by 35%. Adding keyword search over the same enriched passages took the reduction to 49%, and reranking the results took it to 67%. Your numbers will differ. What carries over is that retrieval quality can be measured and improved.

Retrieval also has to keep up. Files change, get replaced and get deleted, so indexing should be incremental and scheduled, with a view of which sources are stale.

Citations turn trust into a one-click check.

Each claim in the answer points to the passage it came from: the file, the page and the words. A reader who doubts a claim opens the source instead of arguing with the assistant. An answer that cannot show its source is a bug, and should be treated as one.

Model providers now support this directly. Anthropic's Citations feature, released in January 2025, returns the passages a response relied on and where they sit in the source documents. Whatever model is used, check citations mechanically before an answer is shown: the quoted words must exist in the cited passage, and the cited file must be one the user may open.

The assistant declines whenever the evidence is thin.

Thin means the retrieved passages do not contain the answer. Set a threshold on the evidence, from retrieval scores and a check that the passages address the question. Below it, the assistant says it does not know and passes the question to a person with what it found. A confident wrong answer does more damage than a plain "I do not know", so set the threshold to lean towards declining.

Test refusal as deliberately as answering. Put questions in the evaluation set whose answers are not in the documents. A system that has never said it does not know has not been tested properly.

Contradictions need their own behaviour. When two documents disagree, such as a conflicting price list or a superseded procedure, the assistant shows both and says that they disagree, which is often more useful than either answer alone.

Access is enforced at retrieval, before the model reads anything.

A passage a person may not see is never fetched for them. Asking the model to be discreet does not hold, and neither does filtering the answer afterwards. If a role may not see a margin, the passage carrying that margin is never retrieved for that role and never enters the model's context. Filtering after the fetch is what leaks a price.

OWASP's 2025 list names the risk directly. In a shared vector store, passages from one group can be retrieved for another. Its mitigation is permission-aware vector stores and strict logical partitioning of the data, which in practice means five rules:

• Copy each file's permissions from its source system onto every passage when it is indexed.
• Filter by the asking user's permissions inside the retrieval query itself.
• Cache answers per permission set as well as per question, or one person's answer can be served to another.
• Re-sync permissions and deletions on a schedule you know, so a revoked share or a deleted file leaves the index.
• Treat retrieved text as untrusted. A document can carry instructions, which is indirect prompt injection, so an assistant that reads documents should not also hold tools that send, pay or delete without a person.

Keep an audit trail of who asked what, what was retrieved and what was answered. You will need it the first time somebody disputes an answer.

Quality is measured on questions your own people write.

Write the set before anyone sees a chat box. A useful one holds a few hundred real questions with agreed answers and sources, a share the documents cannot answer, and questions from each role, so access rules are tested too.

RAGAS, a research framework published in 2023, automates several of these checks. It scores whether the retrieved passages are relevant and focused, whether the answer uses them faithfully, and the quality of the answer itself. Automated scores are a first pass. Calibrate them against answers a person has graded.

Then keep the set. It becomes a regression test that runs on every change, to the model, the prompt, the index or the documents, so an upgrade cannot quietly make answers worse.

Demand these before you trust an assistant.

• Every answer carries the document, the page and the passage.
• It refuses on thin evidence, and you have watched it do so on your own questions.
• It shows conflicting sources instead of silently picking one.
• A role cannot retrieve what it cannot open in the source system, and a test proves it.
• Deleted and re-permissioned files leave the index on a known schedule.
• A question set written by your staff runs on every change.
• An audit trail records the question, the passages and the answer.
• The hosting, the model and the data terms are named in the contract, and none of your documents is used to train a model that others use.

We build every company knowledge base to this list, on hardware you control or on our hosting in Germany.