RAG explained without the hand-waving.
RAG is a simple idea with a lot of engineering hiding underneath: find the right source material first, then answer from that material.
Find the passages most likely to answer the question.
Use those passages as the model's working evidence.
Check the citations or stop when support is too weak.
What RAG actually does
A normal language model answers from patterns learned during training. A RAG system gives it fresh source material at the moment someone asks a question.
The document is split into passages. The system ranks those passages against the question, keeps a small set, and places them beside the question in the model request. The answer can then be tied back to text the user can inspect.
A small example
“Purchases above $500 require written manager approval before the purchase is made.”
“Does a $700 monitor need approval?”
Yes. A $700 purchase needs prior written approval. [1]
“Which monitor brand should I buy?”
The policy does not provide enough information.
Why hallucinations can still happen
RAG lowers the risk of unsupported answers. It does not remove that risk by itself. Failure can enter at several points:
The answer may exist, but chunking or ranking failed to find it.
A related passage can look convincing while still missing the detail the question needs.
Good evidence can still be followed by a sentence the source never said.
Retrieval cannot decide which policy is current unless the system tracks that context.
What a useful test suite checks
- RetrievalDid the relevant passage reach the top results?
- CoverageDoes the evidence support every part of the question?
- GroundingCan every factual claim be traced to a source?
- AbstentionDoes the system refuse questions the source cannot answer?
- Adversarial inputDoes document text fail to override the system's rules?
- User valueDid a person find the answer useful and easy to verify?
How Groundquote handles it
Groundquote uses keyword retrieval, optional semantic retrieval, rank fusion, evidence thresholds, structured model output, and application-level citation checks. If retrieval is weak or a citation is invalid, the answer stops.
You can inspect each part in the public repository or try both an answerable and unanswerable question in the app.