What does the max_tokens parameter control, and what happens when a response hits that limit?

beginner

Re-explain

max_tokens caps how many output tokens the model may generate in one response. It does not limit your input, and it does not change the context window. Anthropic requires it on every request. OpenAI and Gemini treat it as optional.

When the cap is hit, generation just stops, often mid-sentence. You get no error, and you still pay for the tokens produced. So always check the stop field before you use the text:

Anthropic sets stop_reason to max_tokens.
OpenAI sets finish_reason to length.
Gemini sets finishReason to MAX_TOKENS.

Truncated JSON will fail to parse, so treat that value as a real failure.

On reasoning models, hidden reasoning tokens count against the same budget. A tight cap can then return no visible text at all. OpenAI names that parameter max_completion_tokens on Chat Completions and max_output_tokens on the Responses API.

Rewriting in plainer words…

This answer doesn't lend itself to a diagram - it reads best . No credits were charged.

Why there's no diagram: “”

The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓

The diagram below the answer is the concept . Jump to it ↓

Tailored explanation · switch back to · ·

What should the new diagram focus on?

Guided practice for LLM APIs & Integration One question at a time. Answer out loud, get graded, see what you missed.

How well did you know this?

AI:

Saved in this browser - sign in to keep your review list.

How should your speech become text?

Listening… your words appear above as you speak - tap Stop when you're done.

Recording · cr - tap Stop & transcribe when you're done.

Transcribing with AI…

Voice:

Keep going - a few more words and AI can grade it.

Interview lens How interviewers actually use this question

Likely follow-ups

A JSON response comes back truncated. What does your code do next? Check the stop field before parsing. Never patch partial JSON. Retry with a bigger cap or a smaller task, and log how often it happens.
How do you pick a max_tokens value for a given feature? Size it from the longest valid output plus headroom, stay within the model's output limit, and watch the truncation rate in production.
On a reasoning model, how do you set the cap so you still get a visible answer? Hidden reasoning uses the same budget first. Leave room for the visible answer and handle empty text that comes with a max-token stop.
With streaming on, how does the client find out the output was cut off? The stream closes normally and the stop reason arrives in the final events. The client has to read it before treating the text as complete.

What you can say

max_tokens caps how many tokens the model can generate in one response. It limits output only.
It doesn't limit your input or change the context window. Anthropic requires it on every request, and OpenAI and Gemini treat it as optional.
When the model hits the cap, it just stops, often mid-sentence. There's no error, and you still pay for the tokens it produced.
So I check the stop field before I use the text: stop_reason is max_tokens on Anthropic, finish_reason is length on OpenAI, and finishReason is MAX_TOKENS on Gemini.
If I asked for JSON, a truncated response won't parse, so I treat that stop value as a real failure and handle it.
The trap on reasoning models is that hidden reasoning tokens come out of the same budget, so a tight cap can return no visible text at all.

Weak answers to avoid

Says it limits the input or the context window max_tokens caps output only. The context window is a separate limit, so keep the two apart and say which one each setting controls.
Assumes hitting the cap returns an error The call succeeds with cut-off text and you pay for every token. Say you check stop_reason or finish_reason before trusting the output.
Uses max_tokens to make answers shorter A lower cap cuts answers off mid-sentence and doesn't make them concise. Ask for brevity in the prompt and keep the cap as a safety limit.
Forgets that reasoning tokens share the budget On reasoning models a tight cap can go entirely to hidden reasoning and return no visible text. Mention it and leave headroom.

Interview lens

Likely follow-ups, what you can say, and the weak answers to avoid.

Want a quick review of the fundamentals? See the LLM APIs & Integration cheatsheet.

← Back to all LLM APIs & Integration questions

Pro · $10/mo

45 of 52 LLM APIs & Integration answers are in Pro.

Full answers, code samples, and AI explanations that go simpler or deeper. Cancel anytime.

Full answers + code
AI explanations, simpler or deeper
1,000 AI credits / month
Cancel anytime

Get Pro → See plans & pricing →