AI evals are systematic, repeatable measurements of an AI system's quality. Given a set of inputs, how well does the system produce acceptable outputs according to defined criteria? They matter disproportionately in AI engineering because LLM-based systems fail differently from traditional software.

There is no compiler error for a hallucination or stack trace for a subtly wrong answer. Model behavior also varies with prompt wording, model version, and retrieved context.

Without evals, teams rely on "vibe checks": trying a few inputs by hand and shipping when things look fine. That approach cannot detect regressions, cannot compare two prompts objectively, and collapses as the product grows.

With evals, every change (prompt edit, model swap, retrieval tweak) can be scored against a stable dataset before shipping, so iteration becomes fast and safe.

In practice, the quality of a team's eval suite is one of the strongest predictors of the quality of the product itself. Models are increasingly interchangeable; a well-curated eval set encoding what "good" means for your users is a durable, proprietary asset that competitors cannot copy.

Use several eval types, from cheap code checks to careful human review:

• Unit-style assertions: deterministic code checks on outputs. Examples include "response is valid JSON," "contains the refund policy link," "under 200 tokens," and "no personal data regex matches."
• Golden dataset evals: run the system on curated inputs with known reference outputs or labels. Compare via exact match, string similarity, embedding similarity, or a grader.
• LLM-as-judge: a strong model grades outputs against criteria, either standalone (pointwise) or against a reference.
• Pairwise or preference evals: a judge or human picks the better of two outputs (A vs B). Comparing is easier and more reliable than assigning absolute scores.
• Rubric-based evals: scoring against explicit written criteria, one dimension at a time (accuracy, completeness, tone), rather than a single fuzzy "quality" score.
• Human review: expert annotators label outputs. The gold standard for correctness, used to bootstrap datasets and calibrate automated judges.

Mature systems layer all of these: assertions everywhere, judges on samples, humans on a small calibrated slice.

A golden dataset is a curated set of test inputs paired with expected outputs, reference answers, or human-verified scoring criteria. It provides a stable basis for comparing system versions.

Good golden datasets share a few properties. They are representative: drawn largely from real production traffic rather than invented examples, so scores predict real-world behavior. They are diverse: covering common intents, edge cases, adversarial inputs, and known past failures.

They are trusted: every label has been reviewed by someone who understands the domain. A noisy golden set produces noisy scores that teams learn to ignore.

Version every change with a clear reason. This separates real system movement from a score change caused by different test data.

Teams typically start small, around 20 to 100 examples, and grow the set continuously by promoting interesting production failures into it. A small trustworthy set beats a large noisy one.

LLM-as-judge means using a capable language model to evaluate the outputs of an AI system, in place of (or alongside) human reviewers. The judge receives the input, the system's output, and optionally a reference answer and a rubric. It returns a verdict: pass/fail, a score, or a preference between two candidates.

It offers a practical middle ground. Human review is valuable but slow, while code assertions cannot judge qualities such as faithfulness, helpfulness, or tone. A model judge can score a larger sample at lower cost, though the exact speed and price depend on the chosen model.

The critical caveat: a judge is itself a model with biases (position, verbosity, self-preference) and error rates. It must be validated against human labels before its scores are trusted. An uncalibrated judge is just an opinion generator.

Treat the judge prompt as production code: version it, test it, and re-check its agreement with humans over time.

Judge input: user question + system answer + rubric
Judge output: {"verdict": "fail", "reason": "cites a price not present in context"}

Offline evaluation happens before deployment, against a fixed dataset in a controlled environment. You run the candidate system (new prompt, new model, new retrieval config) over a golden set. Score the outputs with assertions, judges, or humans, and compare against a baseline.

Offline evals are reproducible, cheap to repeat, and safe: no user sees a bad output.

Their weakness is coverage; a fixed dataset never fully matches the live input distribution.

Online evaluation happens in production on real traffic. It includes A/B tests comparing variants on business and quality metrics, plus LLM judges scoring sampled live traces asynchronously. It also covers guardrail metrics, user feedback (thumbs, regenerations), and drift monitoring.

Online signals reflect reality, including inputs you never anticipated, but they are noisy, slower to read, and mistakes have user-facing cost.

The two are complementary and form a loop. Offline evals gate releases, while online evaluation finds gaps in that suite. Label useful production failures and add them to the offline set for future releases.

Public benchmarks measure general model capability under standardized conditions; your product needs task-specific evaluation under your conditions. The gap shows up in several ways:

• Distribution mismatch: MMLU (Massive Multitask Language Understanding) is multiple-choice academic knowledge; HumanEval is small self-contained Python functions; SWE-bench is GitHub issue fixing. None resembles your support bot answering questions about your refund policy with your retrieval pipeline and system prompt.
• Contamination: benchmark questions leak into training data, inflating scores in ways that say nothing about generalization to your inputs.
• Saturation: strong models can cluster near a benchmark's ceiling, so small score differences no longer separate them well.
• System vs model: benchmarks test the raw model; your quality depends on the whole system: prompts, retrieval, tools, guardrails, and how they interact.
• Metric mismatch: a model can top LMArena on style preference while being worse at your domain's factual accuracy.

Benchmarks are useful for a first shortlist of candidate models. The decision of which model actually ships should come from running your own eval suite against each candidate.

A trace is the complete, structured record of one request's journey through your AI application. It captures the user input, every intermediate step, and the final output, with timing. A span is a single operation within that trace: one LLM call, one retrieval query, one tool execution, one guardrail check.

Spans nest, so a "handle_question" root span might contain a "retrieve_docs" span and two "llm_call" spans.

Each span carries attributes: model name, prompt and completion (or references to them), input/output token counts, cost, latency, and error status. Spans also carry custom metadata like user tier or feature flag. This structure makes LLM apps debuggable.

When a user reports a bad answer, the trace shows the retrieved documents and final prompt. It also shows what the model returned at each step.

Traces are also the raw material for everything else in the eval stack. Judges score traces, datasets are curated from traces, and cost/latency dashboards aggregate span attributes. Tools like LangSmith, Langfuse, Braintrust, and Arize Phoenix are all built around this trace/span model, increasingly using OpenTelemetry conventions.