LLM APIs & Integration.

An LLM API is a hosted large language model you call over HTTPS. You send a prompt plus settings and get generated text back. The model is stateless, so you resend the conversation or let the API store it.

The first-party APIs to know:

• Anthropic's Messages API, POST /v1/messages.
• OpenAI's Responses API, POST /v1/responses. Chat Completions still works.
• Google's Gemini API, where the Interactions API is now recommended over generateContent.

Cloud platforms resell the same models: Amazon Bedrock, Microsoft Foundry, and Google's Gemini Enterprise Agent Platform, formerly Vertex AI. They add enterprise billing, regions, and cloud identity and access management (IAM).

Many smaller providers ship an OpenAI-compatible endpoint, so one client library often reaches Mistral, DeepSeek or self-hosted models.

A chat request is a stateless list of messages plus a system prompt. Stateless means the server keeps no memory, so you resend the whole conversation each turn.

{
  "model": "claude-sonnet-4-6",
  "max_tokens": 1024,
  "system": "You are a concise Python tutor.",
  "messages": [
    {"role": "user", "content": "What is a decorator?"},
    {"role": "assistant", "content": "A decorator wraps a function..."},
    {"role": "user", "content": "Show me one."}
  ]
}

Roles label who said what. user is your input, and assistant is the model's earlier reply. Replaying past assistant turns is the model's only memory.

The system prompt holds standing instructions: persona, tone, rules. Anthropic takes it as a top-level system field, outside messages. OpenAI puts it inside messages under a system or developer role.

The reply comes back as one assistant message. Append it to messages before the next call.

max_tokens caps how many output tokens the model may generate in one response. It does not limit your input, and it does not change the context window. Anthropic requires it on every request. OpenAI and Gemini treat it as optional.

When the cap is hit, generation just stops, often mid-sentence. You get no error, and you still pay for the tokens produced. So always check the stop field before you use the text:

• Anthropic sets stop_reason to max_tokens.
• OpenAI sets finish_reason to length.
• Gemini sets finishReason to MAX_TOKENS.

Truncated JSON will fail to parse, so treat that value as a real failure.

On reasoning models, hidden reasoning tokens count against the same budget. A tight cap can then return no visible text at all. OpenAI names that parameter max_completion_tokens on Chat Completions and max_output_tokens on the Responses API.

All three send a long-lived secret key in a request header. Only the header name and a few extras differ.

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01"

curl https://api.openai.com/v1/responses \
  -H "Authorization: Bearer $OPENAI_API_KEY"

curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent" \
  -H "x-goog-api-key: $GEMINI_API_KEY"

• Anthropic now takes the key as Authorization: Bearer, and still accepts the x-api-key header shown here. Every request also needs the anthropic-version header, or it fails. The SDK reads ANTHROPIC_API_KEY.
• OpenAI takes Authorization: Bearer, and the SDK reads OPENAI_API_KEY.
• Gemini takes x-goog-api-key, and the SDK reads GEMINI_API_KEY or GOOGLE_API_KEY. Gemini also accepts ?key= in the URL. URLs land in logs and proxies, so prefer the header.

The same models on Amazon Bedrock or Google Cloud use the cloud's own identity and access management (IAM) credentials instead. That means SigV4 request signing or a short-lived OAuth token.

Billing is per token, not per request or per seat. Providers price input and output separately, per million tokens. Input is what you send: the system prompt, the chat history and the new user message. Output is what the model writes, including tool-call arguments, and it usually costs several times more than input.

A token is a piece of text from the model's vocabulary, usually a subword chunk, not a whole word or a character. English prose averages roughly 4 characters per token. Code, JSON and non-English text pack fewer characters into each token, so they cost more.

"tokenization" -> ["token", "ization"]    2 tokens

Each provider uses its own tokenizer, so the same text counts differently across models. Even one provider's count can shift: Claude 4.7 and later produce about 30 percent more tokens for the same text than earlier Claude models.

Streaming sends the answer in small pieces as the model writes them, instead of one block at the end. You turn it on with a request flag.

{
  "model": "claude-opus-4-8",
  "max_tokens": 1024,
  "messages": [{ "role": "user", "content": "Explain streaming." }],
  "stream": true
}

Total generation time does not change. What changes is time to first token, the wait before the first word appears: hundreds of milliseconds instead of many seconds. Users read while the model writes, so the app feels fast.

A long single response can also trip a proxy or gateway timeout. Streaming avoids that, and it lets a user stop a wrong answer early.

The cost is complexity. Usage totals arrive at the end, and a check on the full text, such as JSON validation, cannot run before the user sees it.

The context window is the most tokens a model can hold at once, where a token is a short chunk of text. It covers the system prompt and message history, plus tool definitions and results. Generated tokens count too, so input and output share one budget. Sizes run from roughly 128K tokens to about a million.

If the input alone exceeds it, the request fails before generation with a 400 error, not a shortened answer.

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "prompt is too long: 205000 tokens > 200000 maximum"
  }
}

Two traps. On many models max_tokens counts against the window, so a big value can reject a prompt that fits. Claude 4.5 and newer accept it and stop with model_context_window_exceeded if output fills the window. And quality often drops well before the hard limit, because models recall the middle of a long input less reliably.