An LLM API lets an application send text or other inputs to a hosted model and receive generated content, tool calls, embeddings, or structured data. The provider manages model weights, accelerators, and serving infrastructure.

Engineers commonly encounter APIs from OpenAI, Anthropic, Google, cloud platforms, and services that host open-weight models. Exact model families change quickly, so learn the shared concepts: messages, context limits, streaming, tools, structured output, usage accounting, and rate limits.

Provider details still matter. Authentication, role names, event formats, safety responses, and model features are not identical. Use current official documentation and keep model IDs in configuration. Choose providers by measured quality, latency, cost, data policy, region, and operational support for the product's workload.

A chat request usually contains a model ID and an ordered list of messages. Each message has a role and content. Common roles represent application instructions, user requests, assistant replies, and tool results.

The system or developer instruction defines stable behavior such as scope, tone, and tool policy. User messages contain the current request and data. Previous assistant and user turns may be included when conversation history matters. Most APIs are stateless, so the application resends needed context on each call.

The request can also include output limits, tools, a response schema, and sampling settings. The response contains generated content plus metadata such as finish reason and token usage. Always inspect both, not just the text.

The maximum output token setting places a hard cap on how much the model can generate. Provider names differ, but the idea is the same: generation stops when the limit is reached.

If that happens, the finish reason normally indicates a length limit. The text may end mid-sentence, and JSON or tool arguments may be incomplete. A larger context window does not remove the need for a separate output cap.

Set the limit high enough for the expected response, then ask for a bounded structure or length in the prompt. Check the finish reason before parsing. Retrying with a larger limit may work for a read-only response, but continuing or repeating side-effectful agent work requires careful state handling.

These APIs commonly use a secret API key sent in an HTTP header or handled by the official SDK. Cloud-hosted variants may use service accounts, identity tokens, roles, or workload identity instead.

Keep credentials on the server, never in browser code, mobile bundles, source control, logs, or prompts. Load them from a secret manager or protected environment at runtime. Use different credentials for development, staging, and production.

Apply least privilege where the platform supports it, rotate keys, and monitor usage by project or service. If a key is exposed, revoke it immediately and investigate its activity. Follow current provider documentation for exact header names and cloud authentication flows because they differ.

A token is a small text unit processed by the model. It may be a word, part of a word, punctuation, or whitespace. Images, audio, cached input, and hidden reasoning may have separate token or unit accounting.

Providers usually bill input and output separately, and rates vary by model. Some also distinguish cache writes, cache reads, batch jobs, long contexts, or tool features. Prices change, so do not hard-code them into explanations or business logic.

Read the usage object returned by each request and calculate cost from a versioned price table. Track cost per successful product task, including retries and multi-step calls. Use the model's actual tokenizer or counting endpoint for preflight estimates.

Streaming sends parts of the response as the model generates them instead of waiting for the complete answer. HTTP APIs often deliver these updates through server-sent events.

Users see the first text sooner, which makes a slow generation feel responsive. The application can also display tool progress, usage updates, or status events when the provider exposes them. Total generation time may stay the same.

Streaming adds client work. Chunks may split words or JSON, event types must be assembled correctly, and errors can occur after partial output has been shown. Buffer structured data until it is complete, support cancellation, and mark an answer complete only after the final event and finish reason arrive.

The context window is the maximum token budget for a request. It includes instructions, messages, tool definitions, documents, multimodal inputs, and usually the space needed for the response.

If a request is too large, the API may reject it or a client library may truncate content. Silent truncation is dangerous because it can remove system rules or important evidence. The exact behavior depends on the provider and endpoint.

Count tokens before large requests, reserve output space, and apply an explicit history policy. Common options are dropping irrelevant turns, summarizing old context, retrieving selected documents, and reducing tool descriptions. Log what was removed so a quality failure can be explained.