Temperature controls how random the next-token pick is. It divides the logits, the raw scores the model gives each token, before softmax turns them into probabilities.
probs = softmax(logits / temperature)
A low temperature sharpens the distribution, so the top token wins almost every time. A high temperature flattens it, so unlikely tokens get picked more often. Typical settings fall into three bands:
0 to 0.3 is near-greedy and more repeatable. Use it for extraction, classification and code or tool calls.
0.7 to 1.0 is balanced, for chat, drafting and summaries.
Above 1.2 is very flat, with a risk of incoherent text. Use it for brainstorming or generating diverse candidates.
At exactly 0, implementations skip the division and pick the top token. Raising temperature to escape a wrong answer usually just yields a differently wrong one.
Rewriting in plainer words…
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Next word after "The cat sat on the", at three temperatures
total probability 100 %
T = 0.3, near-greedy99.99 of 100 %
mat (logit 4) 96.43 %
floor (logit 3) 3.44 %
sofa (logit 2) 0.12 %
roof (logit 1) near 0
moon (logit 0) near 0
Dividing the raw scores, called logits, by 0.3 widens the gaps between them, so mat wins 96% of the time.
T = 1.0, balanced100 of 100 %
mat (logit 4) 63.64 %
floor (logit 3) 23.41 %
sofa (logit 2) 8.61 %
roof (logit 1) 3.17 %
moon (logit 0) 1.17 %
Dividing by 1 leaves the logits unchanged. Mat still leads with 64%, but floor and sofa now get real chances.
T = 2.0, very flat100 of 100 %
mat (logit 4) 42.87 %
floor (logit 3) 26 %
sofa (logit 2) 15.77 %
roof (logit 1) 9.56 %
moon (logit 0) 5.8 %
Dividing by 2 shrinks the gaps. Mat falls to 43%. Moon, the word that makes no sense here, gets about 1 in 17 picks.
1step
T = 0.3, near-greedy: Dividing the raw scores, called logits, by 0.3 widens the gaps between them, so mat wins 96% of the time. In all, 99.99 of 100 %.
1 / 3← → keys
Temperature divides the logits before softmax turns them into probabilities. Low values give the top token nearly every pick, and high values spread the odds to unlikely tokens.
Your extraction job returns a wrong field at temperature 0.2. Would you raise the temperature?No. That usually swaps one wrong answer for another. Look at the input instead: prompt, schema, examples or missing context.
What happens to the distribution as temperature goes toward 0, and as it gets very large?Dividing by a small value widens the gaps between logits until the top token takes nearly all the probability. A large value shrinks gaps toward uniform.
You need 20 varied headline ideas. How would you set sampling and keep them usable?Raise temperature or draw several samples for spread, cap the tail with top-p to cut junk tokens, then filter or rank what comes back.
What you can say
Temperature controls how random the model's pick of the next token is.
It divides the logits, the raw score the model gives each token, before softmax turns them into probabilities.
A low temperature sharpens the distribution so the top token almost always wins, and a high one flattens it so unlikely tokens get picked more often.
I'd use 0 to 0.3 for extraction, classification and code, 0.7 to 1.0 for chat and drafting, and above 1.2 only for brainstorming, where text can turn incoherent.
At exactly 0, implementations skip the division and just take the top token.
Raising temperature to escape a wrong answer rarely works, because you usually just get a differently wrong one.
Weak answers to avoid
Calls it a creativity dial and stopsIt names the effect, not the mechanism. Say that it divides the logits before softmax, which sharpens or flattens the distribution.
Suggests raising temperature to fix wrong answersA wrong answer usually comes from the prompt or missing context. More randomness tends to give a differently wrong answer. Fix the input instead.
Gives no settings for real tasksInterviewers want judgment. Tie ranges to jobs: low for extraction and tool calls, around 0.7 to 1.0 for chat, high only for brainstorming.
Says temperature 0 guarantees identical outputGreedy picking is repeatable on paper, but batching and floating-point differences can still flip near-ties. Say more repeatable, not deterministic.
Interview lens
Likely follow-ups, what you can say, and the weak answers to avoid.
