Knowledge in an LLM is stored diffusely across billions of weights, learned from seeing facts many times in varied contexts during pretraining. A fine-tune that mentions a fact a handful of times rarely creates reliably retrievable knowledge. The model may memorize the exact phrasing without generalizing to paraphrased questions.
Worse, research and practice both show fine-tuning on domain data can increase hallucination. This is because the model learns the confident tone and vocabulary of your domain without actually acquiring the underlying facts. So it answers fluently and wrongly.
The operational problems are just as serious:
- Updates: facts change; retraining for every price change or policy update is absurd.
- Deletion: you cannot reliably remove a fact from weights, which is a compliance problem for user data.
- Provenance: no citations, so users cannot verify answers.
- Access control: everyone who can query the model gets the knowledge; per-user permissions are impossible.
RAG solves all four: the index updates instantly, documents can be deleted, answers can cite sources, and retrieval can respect access control.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
The diagram below the answer is the concept . Jump to it ↓