Question 1

What is a golden set in AI evaluation?

Accepted Answer

A golden set is a versioned, frozen, human-owned collection of test items with known expected outcomes. It turns the question of whether an assistant is good into a comparison anyone can re-run: before and after a change, between two models, or month over month. Each item declares an expected behaviour, carries deterministic checks, and has a named owner who answers for it.

Question 2

How many test items does an AI evaluation set need?

Accepted Answer

Twenty to fifty items drawn from real failures is a legitimate starting set, growing to roughly forty to eighty by the time it is frozen. The binding constraint is usually expert time rather than statistics: a twenty-five item human-graded core across a baseline, a four-model comparison, and two improvement cycles is 175 expert judgments, which is three to five weeks of calendar time at typical availability.

Question 3

Can an LLM judge replace human review?

Accepted Answer

Not without calibration. An AI judge should be validated against about a hundred human-labelled outputs before its verdicts count, reported per verdict class with chance-corrected agreement rather than raw percent agreement. Realistic judge-human agreement (Cohen's kappa) sits around 0.3 to 0.5, so judges scale grading while humans remain the gold standard on routed slices.

Question 4

Why does correct refusal matter when evaluating an AI assistant?

Accepted Answer

Most metric libraries score a refusal as a zero or an error, which makes them unsuitable for workplace assistants where declining well is a core competence. A correct refusal and a refusal over an answer that existed are different outcomes with different fixes, and telling them apart means recording what actually reached the assistant rather than reading the refusal alone: if the right document never arrived, the search layer is at fault; if it arrived and the assistant declined anyway, the prompt is.

How do you know your AI assistant works?

What you will learn

Build the instrument

Five verdicts, refusal included

Size it by expert time

Hire the judge

Review without spreadsheets

Freeze, gate, sign