Fine-tuning is the wrong tool because it changes behavior, not storage. It shapes style, format and task shape. Facts land in the weights only as fuzzy statistics, so the model learns the shape of a confident answer and invents the details.
Three concrete problems:
Recall is unreliable. A fact needs many repeated paraphrases before it sticks. A small SFT run rarely gets there.
Facts change. Every correction means another training run and another eval pass. Retrieval needs one document edit.
It can increase hallucination. Training on facts the base model never saw rewards confident guessing, and that habit spreads to other questions.
You also lose citations, per-user access control and instant removal.
As a rule of thumb, use RAG for what the model should know and fine-tuning for how it should answer.
Rewriting in plainer words…
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Saved in this browser - sign in to keep your review list.
How should your speech become text?
Listening… your words appear above as you speak - tap Stop when you're done.
Recording · cr - tap Stop & transcribe when you're done.
Transcribing with AI…
Voice:
Keep going - a few more words and AI can grade it.
Interview lens
How interviewers actually use this question
Likely follow-ups
If fine-tuning is the wrong tool for facts, what is it actually good at?Style, output format, task shape, tool-call syntax, and distilling a large model's behavior into a smaller, cheaper one.
Why would training on facts the base model never saw make it guess more on unrelated questions?The loss rewards a specific, confident answer the model can't tie to anything it knows, so it learns that answer shape even without the knowledge.
How would you test whether a fine-tune made the model hallucinate more?Hold out questions outside the training facts, compare base and tuned models, and count confident wrong answers against honest refusals.
If domain knowledge really has to live in the weights, what would you do instead of a small SFT run?Continued pretraining on a large domain corpus, mixed with general data to limit forgetting, then SFT. It fits stable domains, not facts that change weekly.
What you can say
Fine-tuning changes how a model behaves, not what it reliably knows, so it's the wrong place to put facts.
It shapes style, format and task shape. Facts only land in the weights as fuzzy statistics, so the model learns what a confident answer looks like and makes up the details.
Recall is shaky, because a fact needs many repeated paraphrases before it sticks, and a small SFT run rarely gets there.
Facts also change. Every correction means another training run and another eval pass, while with retrieval I edit one document.
It can also raise hallucination. Training on facts the base model never saw rewards confident guessing, and that habit spreads to other questions.
You also lose citations, per-user access control and instant removal, so my rule is RAG for what the model should know and fine-tuning for how it should answer.
Weak answers to avoid
Claims fine-tuning can't learn facts at allThat overclaims. Facts can stick after many repeated paraphrases, but recall stays unreliable. Say it's the wrong tool, not an impossible one.
Only talks about retraining costUpdate cost is one reason. The stronger points are unreliable recall and that training on unseen facts rewards confident guessing, which raises hallucination.
Says use RAG and stops thereNaming RAG isn't an explanation. Say fine-tuning shapes behavior, then say what retrieval adds: one-document edits, citations, per-user access control and removal.
Says domain fine-tuning reduces hallucinationOften the reverse. Tuning on facts the base model never saw teaches confident guessing, and that spreads beyond the trained facts.
Interview lens
Likely follow-ups, what you can say, and the weak answers to avoid.
