LearnThatStack Ace your next interview
LLM APIs & Integration · question
Question 7 of 52

What is a context window, and what happens if you exceed it?

beginner
← All LLM APIs & Integration questions
Re-explain
Visual ↓

The context window is the most tokens a model can hold at once, where a token is a short chunk of text. It covers the system prompt and message history, plus tool definitions and results. Generated tokens count too, so input and output share one budget. Sizes run from roughly 128K tokens to about a million.

If the input alone exceeds it, the request fails before generation with a 400 error, not a shortened answer.

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "prompt is too long: 205000 tokens > 200000 maximum"
  }
}

Two traps. On many models max_tokens counts against the window, so a big value can reject a prompt that fits. Claude 4.5 and newer accept it and stop with model_context_window_exceeded if output fills the window. And quality often drops well before the hard limit, because models recall the middle of a long input less reliably.

Rewriting in plainer words…

This answer doesn't lend itself to a diagram - it reads best . No credits were charged.

Why there's no diagram: “”

The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓

The diagram below the answer is the concept . Jump to it ↓

Tailored explanation · switch back to · ·
What should the new diagram focus on?
Guided practice for LLM APIs & Integration One question at a time. Answer out loud, get graded, see what you missed.
See it - interactive diagram
What fills a 200K context window
Context window 200K tokens
Fits: small max_tokens186K of 200K tokens
  • System prompt 5K tokens
  • Tool definitions 10K tokens
  • Message history 120K tokens
  • Tool results 35K tokens
  • Output max_tokens 16K

The system prompt, tool definitions, history and tool results take 170K tokens. max_tokens caps the reply at 16K, so both fit.

Same prompt, big max_tokens234K tokens, over the limit
  • System prompt 5K tokens
  • Tool definitions 10K tokens
  • Message history 120K tokens
  • Tool results 35K tokens
  • Output max_tokens 64K

The prompt fits, but max_tokens is now 64K. Many models count that against the window, and 234K is over 200K, so they reject it.

Claude 4.5+, big max_tokens200K of 200K tokens
  • System prompt 5K tokens
  • Tool definitions 10K tokens
  • Message history 120K tokens
  • Tool results 35K tokens
  • Output stops at limit

Claude 4.5 and newer accept this request. The model stops writing when the window fills and returns the stop reason model_context_window_exceeded, not an error.

Input alone too long205K tokens, over the limit
  • System prompt 5K tokens
  • Tool definitions 10K tokens
  • Message history 155K tokens
  • Tool results 35K tokens

The history grows to 155K, so the input alone is 205K tokens. The API returns a 400 error before writing anything, not a shorter answer.

Input and output share one window, so the API can reject a prompt that fits once you add a big max_tokens on top.

© LearnThatStack - diagrams may not be republished without permission.

Other questions have diagrams like this too. Sign in to ask for one where it's missing - it's free.

Create a free account
How well did you know this?
AI:

Saved in this browser - sign in to keep your review list.

How should your speech become text?

Listening… your words appear above as you speak - tap Stop when you're done.

Recording · cr - tap Stop & transcribe when you're done.

Transcribing with AI…

Voice:

Keep going - a few more words and AI can grade it.

Interview lens How interviewers actually use this question

Likely follow-ups

  • A chat keeps growing past the window. How do you manage history without breaking prompt caching? Name trimming, summarizing old turns or clearing old tool results, and keep the cached prefix stable so every change lands after it.
  • How do you know a request will fit before you send it? Use the provider's own token counting endpoint or tokenizer, not a generic one, and budget for tool definitions and max_tokens too.
  • A model takes a million tokens. Why not put the whole knowledge base in every prompt? Cost and latency grow with input tokens, and recall drops in long inputs. Retrieve the relevant chunks and cache the stable prefix.
  • In an agent loop, what fills the window fastest, and how do you keep it under control? Tool results pile up every turn. Truncate or summarize big results, drop stale ones, and keep tool definitions short.

What you can say

  1. The context window is the most tokens a model can hold in one request, and a token is a short chunk of text.
  2. Everything counts against it: the system prompt, the message history, tool definitions and results, and the generated tokens. Input and output share one budget.
  3. Sizes run from about 128K to around a million tokens. If the input alone is over the limit, the request fails with a 400 error before generation. You don't get a shortened answer.
  4. One trap is max_tokens. On many models it counts against the window, so setting it high can get a prompt that fits rejected.
  5. Claude 4.5 and newer accept that request and stop with a model_context_window_exceeded stop reason if the output fills the window.
  6. I wouldn't treat the limit as the real ceiling, though. Quality often drops well before it, because models recall the middle of a long input less reliably.

Weak answers to avoid

  • Thinks only the prompt counts Output tokens, tool definitions and tool results share the same budget. Say that on many models a big max_tokens can push a prompt that fits over the limit.
  • Says the API trims an oversized input The API rejects it with a 400 invalid_request_error before generating anything. Trimming history is your app's job, and you have to build it.
  • Treats the advertised size as usable capacity Recall of material in the middle of a long input drops well before the hard limit. Say you keep prompts lean instead of filling the window.
  • Confuses the window with memory across chats The window covers one request. The app resends history each turn, and that growing history is what eventually hits the limit.
Interview lens

Likely follow-ups, what you can say, and the weak answers to avoid.

Sign in free to open it Free account - the lens opens as soon as you're back.

Want a quick review of the fundamentals? See the LLM APIs & Integration cheatsheet.

← Back to all LLM APIs & Integration questions
Pro · $10/mo

45 of 52 LLM APIs & Integration answers are in Pro.

Full answers, code samples, and AI explanations that go simpler or deeper. Cancel anytime.

  • Full answers + code
  • AI explanations, simpler or deeper
  • 1,000 AI credits / month
  • Cancel anytime