Prompt engineering without an eval set is just taste. It works fine until two people on the team have different taste, or until the model updates and nobody can prove whether things got better or worse.
The objection is always time. So build the smallest set that is still useful: forty examples.
What goes in the forty
Not forty random samples. A deliberate spread:
- 20 ordinary cases — what users actually do most of the time
- 10 edge cases — empty input, very long input, wrong language, ambiguous requests
- 10 known failures — every bug report you have received, frozen as a test
The last group is the one that pays for the whole exercise. A bug that is written down as an eval case cannot silently come back.
Grade the thing you care about
Most teams reach for an LLM judge immediately. Try cheaper checks first, because they are deterministic and free:
def grade(case, output):
if case.kind == "extraction":
return output.json == case.expected # exact match
if case.kind == "refusal":
return output.refused is case.should_refuse # boolean
if case.kind == "citation":
return all(c in case.allowed for c in output.citations)
return judge(case, output) # LLM, last resort
Reserve the LLM judge for genuinely subjective dimensions — tone, helpfulness, whether an explanation is clear. And when you use one, validate it: grade thirty outputs by hand, compare with the judge, and measure the agreement. A judge you have not checked is a number you cannot use.
Run it on every change
The set only earns its keep if running it is trivial. One command, under two minutes, results committed alongside the prompt. When the numbers live in the repo, a prompt change becomes reviewable like any other diff:
accuracy 0.88 → 0.91 (+0.03)
refusal rate 0.04 → 0.11 (+0.07) ← look at this
p95 latency 740ms → 760ms
That refusal jump is the kind of regression that ships silently when nobody is measuring. Here it shows up in review.
An eval set is not a quality gate. It is a way to have arguments with evidence.
Start today
Pick your last ten bug reports. Write them as cases. You now have a quarter of an eval set, and every one of those cases is a real failure you already paid for once.