LearnThatStack Ace your next interview
LLM APIs & Integration · question
Question 2 of 52

Walk me through the anatomy of a basic chat request: messages, roles, and the system prompt.

beginner
← All LLM APIs & Integration questions
Re-explain

A chat request is a stateless list of messages plus a system prompt. Stateless means the server keeps no memory, so you resend the whole conversation each turn.

{
  "model": "claude-sonnet-4-6",
  "max_tokens": 1024,
  "system": "You are a concise Python tutor.",
  "messages": [
    {"role": "user", "content": "What is a decorator?"},
    {"role": "assistant", "content": "A decorator wraps a function..."},
    {"role": "user", "content": "Show me one."}
  ]
}

Roles label who said what. user is your input, and assistant is the model's earlier reply. Replaying past assistant turns is the model's only memory.

The system prompt holds standing instructions: persona, tone, rules. Anthropic takes it as a top-level system field, outside messages. OpenAI puts it inside messages under a system or developer role.

The reply comes back as one assistant message. Append it to messages before the next call.

Rewriting in plainer words…

This answer doesn't lend itself to a diagram - it reads best . No credits were charged.

Why there's no diagram: “”

The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓

The diagram below the answer is the concept . Jump to it ↓

Tailored explanation · switch back to · ·
What should the new diagram focus on?
Guided practice for LLM APIs & Integration One question at a time. Answer out loud, get graded, see what you missed.
How well did you know this?
AI:

Saved in this browser - sign in to keep your review list.

How should your speech become text?

Listening… your words appear above as you speak - tap Stop when you're done.

Recording · cr - tap Stop & transcribe when you're done.

Transcribing with AI…

Voice:

Keep going - a few more words and AI can grade it.

Interview lens How interviewers actually use this question

Likely follow-ups

  • The server keeps no memory. What does that do to cost and latency as a chat gets longer? Input tokens grow every turn because the full history is resent. Name caching the stable prefix as the usual fix.
  • What do you do once the conversation no longer fits in the context window? Trim or summarize old turns, keep the system prompt and recent turns, and keep the start of the prompt stable so caching still hits.
  • If the client resends history, could a user forge an earlier assistant turn? How do you stop that? Yes, the API trusts whatever history it receives. Keep the transcript on your server and never accept it from the browser as is.
  • Why put rules in the system prompt instead of the first user message? It keeps standing rules apart from user input, models give it priority as instructions, and a fixed one is a stable prefix for caching.
  • Beyond the system field, how do Anthropic, OpenAI and Gemini request shapes differ? Gemini uses contents with parts and a model role, Anthropic requires max_tokens, and tool calls and results sit in different places.

What you can say

  1. A chat request is a list of messages plus a system prompt, and the API is stateless, so the server remembers nothing between calls.
  2. Each message has a role, where user is what I send and assistant is a reply the model gave earlier.
  3. Because nothing is stored, I resend the whole conversation every turn, and those replayed assistant turns are the model's only memory.
  4. The system prompt holds standing instructions like persona, tone and rules, and on Anthropic it goes in a top-level system field outside messages.
  5. OpenAI puts the same thing inside messages under a system or developer role, so the idea is the same and only the placement differs.
  6. The reply comes back as one assistant message that I append before the next call, and since every call resends it all, input tokens grow with each turn.

Weak answers to avoid

  • Thinks the server remembers the chat The API is stateless. Send only the newest message and the model has no context. Say you resend the whole conversation every turn.
  • Assumes every API places the system prompt the same way Anthropic takes it as a top-level system field. OpenAI puts it in messages with a system or developer role. Naming that difference shows you have used both.
  • Forgets to append the assistant reply If you drop the model's reply, the next turn can't see what it said. Append each reply to messages before the next call.
  • Recites the JSON without saying why roles matter Field names alone sound memorized. Say that replayed assistant turns are the model's only memory, which is why the roles have to be right.
Interview lens

Likely follow-ups, what you can say, and the weak answers to avoid.

Sign in free to open it Free account - the lens opens as soon as you're back.

Want a quick review of the fundamentals? See the LLM APIs & Integration cheatsheet.

← Back to all LLM APIs & Integration questions
Pro · $10/mo

45 of 52 LLM APIs & Integration answers are in Pro.

Full answers, code samples, and AI explanations that go simpler or deeper. Cancel anytime.

  • Full answers + code
  • AI explanations, simpler or deeper
  • 1,000 AI credits / month
  • Cancel anytime