An eval, short for evaluation, is a repeatable test that runs your AI system over a fixed set of inputs and scores the outputs. It is measurement, not a one-off demo.
case: "refund request, order already shipped"
input: user message + order state
expect: mentions the 30-day window, does not promise a refund
score: pass / fail
It is called the core discipline because the model itself is bought, not built. Everyone can call the same API. What you own is your definition of "good" for your task, plus proof that a change helped.
Without evals, every prompt or model change is a guess judged by feel. With them, shipping becomes a loop: measure, change one thing, measure again.
Evals also fail differently from normal tests. There is no compiler error. A bad output looks perfectly confident, so someone must define correctness on purpose.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
The diagram below the answer is the concept . Jump to it ↓