home
›
cheatsheets
›
ai_engineering
›
llm_api_integration
Free to read Every card and diagram, and Quiz me
2
Free account Star cards. Your Quiz me marks are kept, so next time you drill just what you missed
3
Pro All 52 answers on this sheet, My sheet across every topic, and PDFs
A free account keeps your stars and Quiz me marks. Pro adds all 52 answers on this sheet, My sheet across every topic, and PDFs.
/
Read
Quiz me
Comfortable
Compact
0
Click any code to copy it
Done 0 /0
To review 0
On this sheet
01 Requests & auth02 Tokens & cost03 Output, sampling & reasoning04 Streaming05 Tools & MCP06 JSON & schemas07 Images, PDFs & audio08 Caching & batch09 Errors, retries & limits10 Routing & production
The model An LLM API is stateless . Every call carries the system prompt, the whole conversation and the tools. You append the model's reply before the next call. You pay per token , at separate prices for input and output. Output costs several times more, and hidden reasoning tokens bill as output. Read the stop reason before the text. A reply cut off by max_tokens and a tool call both come back as HTTP 200, with no error. The model never runs your code. It asks for a tool call , and your server runs it and sends back the result. The loop repeats until the model stops asking. Prompt caching matches an exact prefix . Keep the tools and system prompt fixed at the front, and put whatever changes at the end. The model keeps no memory, so your server resends the whole history every call. It runs each tool the model asks for and loops until a reply ends in end_turn.
Requests & auth the call every feature starts with Pick a provider API Call each provider's API POST /v1/messages POST /v1/responses POST /v1beta/models/{model}:generateContent Anthropic, OpenAI, then Gemini. Clouds like Bedrock resell the same models.
Choose the official SDK or raw HTTP The SDK. It handles auth, retries, timeouts and streams.
Send a turn with OpenAI's Responses API client.responses.create(model=m, input=q) Send input, get a typed output list.
Continue a chat without resending it previous_response_id=r1.id OpenAI keeps the history.
Replace the Assistants API Use Responses. OpenAI removed it on August 26, 2026.
Tip Chat Completions still works. Keep it for code that also targets other providers.
Build a chat request {"model": "claude-sonnet-4-6", "max_tokens": 1024, required "system": "You are a concise Python tutor.", "messages": [ {"role": "user", "content": "What is a decorator?"}, {"role": "assistant", "content": "It wraps a..."}, sent again {"role": "user", "content": "Show me one."}]}
Give the model memory of earlier turns Resend the whole chat, with each reply appended.
Set the system prompt on OpenAI or Gemini "role": "developer" "systemInstruction" OpenAI, then Gemini.
Find the reply in the response content on Anthropic, choices on OpenAI, candidates on Gemini.
Trap Anthropic puts text and tool calls in one block list. OpenAI splits them into content and tool_calls.
Authenticate and keep the key safe Send the API key x-api-key Authorization: Bearer x-goog-api-key Anthropic, OpenAI, then Gemini.
Add Anthropic's version header anthropic-version: 2023-06-01 Every request fails without it.
Add LLM features to a frontend Call your own server. It checks the user and holds the key.
Trap Next.js inlines NEXT_PUBLIC_ variables into the bundle. Anyone can read a key sent to the browser and spend your money.
Tokens & cost what you pay for, and how to count it Tokens and the context window Know what you pay for Input and output tokens. Output costs several times more.
Guess a token count from text One token is about 4 English characters. Code and JSON take more.
Know what fills the context window Prompt, history, tools and the output, in one budget.
Send more input than the window holds It fails with a 400 before generating. Quality drops well before that.
Trap On many models max_tokens counts against the window. A large value can reject a prompt that fits.
Count tokens and read usage Count a Claude request client.messages.count_tokens(**req) Free, on the server. Close to the bill.
Count an OpenAI or Gemini request tiktoken.encoding_for_model(m) client.models.count_tokens(**req) tiktoken is local and close. Gemini's is exact.
Price an Anthropic call cache_read_input_tokens cache_creation_input_tokens Outside input_tokens. Add each at its own rate.
Price an OpenAI call prompt_tokens_details.cached_tokens Inside prompt_tokens. Subtract, then price the rest.
Get usage from an OpenAI stream stream_options={"include_usage": True} It comes on the final event.
Trap Counts don't carry across models. tiktoken is wrong for Claude, and Claude 4.7+ counts about 30% more than older Claude.
Output, sampling & reasoning cap, steer and check what the model writes Cap output and check why it stopped Cap how much the model writes "max_tokens": 1024 max_completion_tokens=1024 "maxOutputTokens": 1024 Anthropic, OpenAI, then Gemini. Only Anthropic requires it.
Spot a reply the cap cut off stop_reason: "max_tokens" finish_reason: "length" finishReason: "MAX_TOKENS" Anthropic, OpenAI, then Gemini.
Trap Hitting the cap raises no error. You still pay for the tokens, and cut-off JSON won't parse.
Sampling and reasoning effort Make output more or less random "temperature": 0.2 "top_p": 0.9 Temperature scales the odds. top_p keeps the likeliest tokens. Tune one.
Set temperature on a reasoning model Claude 4.7+ returns a 400. GPT-6 allows it only at effort none.
Set reasoning effort on Claude "thinking": {"type": "adaptive"} "output_config": {"effort": "high"} Pass thinking blocks back unchanged in a tool loop.
Set reasoning effort on OpenAI or Gemini reasoning_effort="high" thinking_level="high" OpenAI, then Gemini 3. It replaces thinking_budget.
Tip Leave room in max_tokens. Thinking bills as output and shares the cap.
Trap temperature: 0 doesn't make output repeatable. Batching and floating-point effects still vary it.
Streaming show text while the model writes it Turn on streaming No stream model writes all text Stream model writes first words time Both replies take the same time to write. Each dot is text reaching the screen. Streamed text starts almost at once, so the user reads as the model writes.
Stream a reply "stream": true
Say why chat apps stream First words in under a second. It also avoids proxy timeouts.
Trap Usage arrives at the end, and you can't validate the full text before the user sees it.
Read the event stream event: message_start input tokens event: content_block_start one per block event: content_block_delta repeats event: content_block_stop event: message_delta stop_reason, usage event: message_stop
Parse one server-sent event An event: line, a data: line of JSON, then a blank line.
Rebuild tool arguments from a stream input_json_delta Join the pieces. Parse when the block stops.
Trap A ping or an error event can arrive at any point. An error cuts the stream short.
Handle the stream in the client Keep characters whole across chunks decoder.decode(chunk, { stream: true })
Update the UI without stalling the tab requestAnimationFrame(paint) One paint per frame, not per chunk.
Stop a reply mid-stream controller.abort() Call it from a stop button or an idle watchdog.
Prove a stream finished message_stop data: [DONE] Anthropic, then OpenAI. Nothing else proves it.
Recover a long answer that broke off Send the partial text back as an assistant turn. Ask for the rest.
Tip A chunk is any slice of bytes. Keep one buffer per block, and skip event types you don't know.
Trap A failure mid-stream comes after status 200. Drop half-built tool JSON, never run it.
JSON & schemas get output your code can parse JSON mode and structured outputs Get valid JSON from OpenAI {"type": "json_object"} Your messages must mention JSON.
Enforce a schema on OpenAI {"type": "json_schema"} "strict": true
Enforce a schema on Anthropic output_config.format Or a strict tool. Claude 4.6+ rejects a prefilled {.
Get JSON from Gemini responseMimeType responseSchema The first gives JSON. The second adds a schema.
Tip Structured outputs mask every token that would break the schema. Asking in the prompt can't promise that.
Trap JSON mode fixes the syntax only. Check key names, types and required fields yourself.
Where schemas still fail Write a schema the provider accepts "additionalProperties": false Only a subset works. Anthropic's SDKs strip minimum and maxLength.
Handle an object cut off halfway Check the stop reason first, and raise the budget. Don't retry the parse.
Handle a refusal It returns prose. Use a fallback path, not a repair loop.
Catch valid JSON with wrong values Check business rules in code. Feed the error back once, then dead-letter it.
Images, PDFs & audio send more than text Send images and PDFs Put an image in a message "type": "image" "type": "input_image" inline_data Anthropic, OpenAI, then Gemini.
Keep image costs down Resize first. Claude shrinks past a 1568px edge, or 2576px on 4.7+.
Put a PDF in a message "type": "document" "type": "input_file" "mime_type": "application/pdf" Anthropic, OpenAI, then Gemini. Each reads pages as images and text.
Know the PDF limits Claude takes 32 MB and up to 600 pages. OpenAI takes 50 MB and 100, Gemini 50 MB and 1,000.
Trap Base64 grows the payload by about a third. Upload big or repeated files once through a Files API.
Take voice input Build voice input that works with any model Transcribe first, then send plain text. Claude takes no audio.
Transcribe on OpenAI POST /v1/audio/transcriptions gpt-transcribe, which replaces whisper-1.
Send audio straight to the model Gemini takes it, at 32 tokens a second. Use it when tone matters.
Tip For two-way voice, use OpenAI's Realtime API or Gemini's Live API.
Caching & batch pay less for repeated and offline work Cache a prompt prefix Cache everything up to a block on Anthropic "cache_control": {"type": "ephemeral"} Up to four breakpoints, or one at the top level.
Price a cache write and a read A write costs 1.25x input, or 2x for the 1-hour TTL. A read costs 0.1x or less.
Know when caching pays off By the second call within 5 minutes, or the third on the 1-hour TTL.
Cache on OpenAI prompt_cache_key="support-bot" Automatic, with no storage fee.
Cache on Gemini Automatic on 2.5+. Explicit caches bill hourly storage.
Trap One changed byte early in the prefix misses everything after it. The order is tools, then system, then messages.
Keep long chats cached Trim a long chat without losing the cache Compact old turns into a frozen summary up front. Append new turns.
Cache the growing conversation Move the breakpoint to the newest message each turn.
Share one cache across tenants Shared prompt first, then tenant data, then turns. An org per tenant splits it.
Trap A sliding window that drops the oldest turn changes the prefix, so every request misses.
Run a batch job Pick work for a batch Evals, bulk labels, backfills. Half price, results within 24 hours.
Match results to requests "custom_id": "row-1" Results come back in any order.
Trap A batch can succeed while some requests in it fail. Check every result's status.
Errors, retries & limits survive 429s, timeouts and outages Handle a 429 and plan capacity Wait out a rate limit retry-after Wait that long. With no header, back off with jitter.
Spot a 429 that never clears credit_balance_exhausted enforced_spend_limit_reached OpenAI, then Anthropic. Fail fast.
Name the rate limits Requests and tokens per minute, daily caps and concurrency.
Budget tokens per minute on OpenAI Peak requests times input plus max_tokens. OpenAI counts it up front.
Retries and timeouts Pick the errors to retry Timeouts, network errors, 429s and 5xx. Never a 400, 401, 403 or 404.
Space out retries random.uniform(0, min(cap, base * 2 ** n)) Full jitter. Stop after about three tries.
Run your own retry loop Anthropic(max_retries=0) SDKs already retry twice. Don't stack layers.
Tighten the SDK defaults Anthropic(timeout=30.0, max_retries=1) Defaults are 10 minutes and 2 retries, so up to 30 minutes.
Beat the platform's own timeout Proxies and serverless cut at 30 or 60 seconds. Set yours lower.
Put a time limit on a long stream The read timeout counts gaps between chunks. Add an overall deadline.
Trap A client timeout doesn't stop the model. You can pay for tokens you never read.
Layer retries, breakers and fallbacks deadline: the whole user request, like 30s fallback: primary, other provider, cached breaker per provider, model, region hedge: a copy at p95, first reply wins retry: jittered, from a shared budget timeout per attempt, inside the rest
Make a tool call safe to retry Key it on session, turn, tool and arguments. Claim the key before acting.
Decide what trips the circuit breaker Timeouts, 5xx and resets. A 429 waits once and doesn't count.
Trap A resampled response gets a fresh tool_use_id, so never key side effects on it.
Routing & production run LLM calls across providers and at scale Route and fall back across providers Route requests to models By task class, each with an ordered list of pinned model IDs.
Pick what triggers a fallback 429s, 5xx, timeouts and bad schemas. Never a 400, which fails everywhere.
Keep a conversation on one model Route by conversation ID. A switch drops the cached prefix.
Tip One wrapper for every provider shrinks to their shared features. The rest leaks back as if branches.
Trap A fallback starts with a cold cache and another tool-call shape. Test its prompt and parser in CI.
LiteLLM or OpenRouter Run your own gateway The LiteLLM proxy, with virtual keys and budgets per team.
Reach hundreds of models with one key and one bill OpenRouter, a hosted OpenAI-compatible gateway. It charges a fee on credits.
Go direct instead of through a gateway For zero data retention, a BAA, spend discounts or new features first.
Trap A gateway sits in your hot path. Run two replicas or more, and keep its timeouts inside yours.
Pin models and migrate Pin a model in production claude-haiku-4-5-20251001 A snapshot is frozen. An alias like -latest moves.
Tell deprecation from retirement Deprecation is the notice. Retirement is when the ID starts failing.
Move to a new model Canary it against a golden eval set. Recheck cost, tools and output shapes.
Log and deploy in production Log an LLM call One record per HTTP attempt, with request ID, served model, stop reason and usage.
Watch the fleet's health Time to first token, latency, errors by status, cache hits and cost.
Set SLOs for an LLM product Time to first token, inter-token latency and cost per completed task.
Log usage from a serverless function waitUntil(logUsage(resp)) Or await it. The instance freezes once you respond.
Run an agent loop that takes minutes Run it as a durable job. Return a run ID, and checkpoint each step.
Trap A truncated reply returns HTTP 200. Alert on stop_reason, or it looks healthy in every error metric.
Keys across teams Give each team a key A virtual key from your gateway. Real keys stay in a secrets manager.
Scope the real keys One project or workspace per team and environment, with its own limits.
Drop static keys Anthropic's workload identity federation issues short-lived tokens.
Trap Never log the Authorization header.
Nothing matches. Try a shorter word, or clear the search.
Practice Guided practice One question at a time, with the ones you miss coming back. Practice 52 interview questions Full answers, from beginner to expert.
Found this useful? Pass it on.
Share on LinkedIn
Share on X
Copy link
This sheet stays free. With Pro , you can:
Every answer on this sheet. Read the full answer to all 52 questions it links, not just the 7 free ones.My sheet. The cards you star and the ones you miss in Quiz me, from every sheet, on one page to quiz and print.PDFs. Download this sheet and My sheet, laid out for A4, with every question a link to its answer.The rest of Pro. Every answer in the question library, and 1,000 AI credits a month.
