What is the difference between pretraining and post-training?

beginner

Re-explain

Pretraining builds general knowledge, and post-training teaches the model how to behave. In pretraining, the model learns to predict missing or next tokens across a huge dataset. The result is a base model that may only continue text, not answer requests.

Post-training then shapes that base model in stages:

Supervised fine-tuning (SFT) trains on written examples of good answers.
Preference optimization, through RLHF or DPO, then trains on ranked answer pairs that people prefer.

Pretraining takes far more data and compute than post-training. But post-training can add biases like excessive agreement or refusal.

Rewriting in plainer words…

This answer doesn't lend itself to a diagram - it reads best . No credits were charged.

Why there's no diagram: “”

The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓

The diagram below the answer is the concept . Jump to it ↓

Tailored explanation · switch back to · ·

What should the new diagram focus on?

Guided practice for LLM Fundamentals One question at a time. Answer out loud, get graded, see what you missed.

How well did you know this?

AI:

Saved in this browser - sign in to keep your review list.

How should your speech become text?

Listening… your words appear above as you speak - tap Stop when you're done.

Recording · cr - tap Stop & transcribe when you're done.

Transcribing with AI…

Voice:

Keep going - a few more words and AI can grade it.

Interview lens How interviewers actually use this question

Likely follow-ups

Why isn't supervised fine-tuning enough on its own? What does preference optimization add? A strong answer says SFT copies fixed examples, while ranking two answers is easier for people and teaches the model which output is better.
How does DPO differ from PPO-based RLHF, and why might a team pick it? A strong answer says DPO trains on preference pairs with no separate reward model or RL loop, and names the cost and stability gain.
What failure modes can post-training introduce, and where do they come from? Name sycophancy and over-refusal, tie them to what human raters reward, and mention reward hacking when a learned reward model is the target.
When would you pick a base model over an instruct model? Say instruct models follow requests, while base models suit raw completion, few-shot pattern prompts, and teams that run their own post-training.
Your team wants the model to know internal product facts. Is post-training the right tool? Say fine-tuning mostly shapes behavior and format, while facts that change or must be cited usually belong in retrieval.

What you can say

Pretraining builds the model's general knowledge, and post-training teaches it how to behave.
In pretraining, the model learns to predict the next or missing token across a huge dataset.
What comes out is a base model, which may just continue your text instead of answering your request.
Post-training shapes that base model in stages, starting with supervised fine-tuning on written examples of good answers.
Then preference optimization, through RLHF or DPO, trains it on pairs of answers ranked by which one people prefer.
Pretraining takes far more data and compute, but post-training has its own risk, since it can add biases like agreeing too much or refusing too often.

Weak answers to avoid

Calls post-training more pretraining on new data That misses the purpose. Say it trains on example answers and ranked answer pairs to change how the model behaves, while the knowledge comes from pretraining.
Mentions RLHF but skips SFT Post-training runs in stages. Name supervised fine-tuning on example answers first, then preference optimization with RLHF or DPO.
Presents post-training as pure upside It can make a model agree too readily or refuse too often. Name that cost, since it explains behavior you will see in production.
Can't say what a base model does Many candidates assume a base model answers questions. Say it may only continue the text, which is the reason post-training exists.

Interview lens

Likely follow-ups, what you can say, and the weak answers to avoid.

Want a quick review of the fundamentals? See the LLM Fundamentals cheatsheet.

From the blog: Jev explained: the AI model that returns decisions, not text

← Back to all LLM Fundamentals questions

Pro · $10/mo

51 of 58 LLM Fundamentals answers are in Pro.

Full answers, code samples, and AI explanations that go simpler or deeper. Cancel anytime.

Full answers + code
AI explanations, simpler or deeper
1,000 AI credits / month
Cancel anytime

Get Pro → See plans & pricing →