A golden dataset is a curated set of test inputs paired with expected outputs, reference answers, or human-verified scoring criteria. It provides a stable basis for comparing system versions.
Good golden datasets share a few properties. They are representative: drawn largely from real production traffic rather than invented examples, so scores predict real-world behavior. They are diverse: covering common intents, edge cases, adversarial inputs, and known past failures.
They are trusted: every label has been reviewed by someone who understands the domain. A noisy golden set produces noisy scores that teams learn to ignore.
Version every change with a clear reason. This separates real system movement from a score change caused by different test data.
Teams typically start small, around 20 to 100 examples, and grow the set continuously by promoting interesting production failures into it. A small trustworthy set beats a large noisy one.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
The diagram below the answer is the concept . Jump to it ↓