ENTERPRISE AI

Evals and release gates

A golden set is your real cases, run again after every meaningful change. A model judge has to be checked against people. Public benchmarks are not your gate.

Figure 1: A golden set of real cases, a release gate, and a wider release only after the evidence.

1. The output moves, so one lucky demo is not a test

Model output is not deterministic in the way a pure function is. The same prompt can vary. A release gate that is “it looked good in the demo” will pass a regression you have not noticed yet. An eval is the set of cases you run again, on purpose, after a change to a prompt, a model, a skill, or the retrieval.

Build the set from real inputs and the outcome a domain expert expects. Add every production failure as a permanent case. Start by reading the failures you already have, group them by cause, and write the criteria from those groups. Criteria invented before you have seen a failure tend to miss the failure.

2. A golden set, then a gate, then a wider release

A golden set is the curated cases you trust as the repeatable check. It is not the internet, and it is not a public leaderboard. Public benchmarks say something about broad capability. They do not say whether the model behaves with your data, your tools, and your rules.

The gate is: this change does not ship if the set got worse beyond a limit you wrote down before you looked at the result. Shadow, canary, and full release are the longer path in the hardening masterclass. This page is the reason that path exists. Ship the evidence, not the hope.

3. A judge that nobody checked is a preference

Some qualities need a person: faithfulness, a nuanced policy, whether the answer actually helped. That does not scale, so teams add a model as a judge. A judge has to be validated. Hand-label a sample. See whether the judge agrees. Tighten the rubric until it does. An unvalidated judge can be consistently wrong. One published failure mode is preferring the longer answer.

Run the judge separately, so it is not grading its own homework. Write a specific rubric. “Does it cite the refund policy, and is the citation in the packet?” is a rubric. “Is it good?” is a mood. Use a deterministic check wherever the question is a format, a schema, or a required field. Use a behavioral check, from the trace, when the question is whether the right tool was called.

Layman rule: If you did not compare the judge to a person, you do not have a judge. You have a second opinion from the same family of models.

4. Recalibrate, because the world moves

Inputs change. Policies change. A set that matched production in March can miss May. Sample production, review the slices that matter, and refresh the cases. Critical slices deserve their own score, not only an average that hides them. Unknowns deserve a score too: how often the system abstains, and how often that abstention was right.

Operational outcomes belong in the eval, not only text quality. Did the workflow finish? Did a person have to repair it? A beautiful answer that did not complete the task is a failed task.

5. What to count next

Write the limit before the next change. Run the set. If you cannot say whether the change was better, you are not ready to release it.

Better is not the same as cheaper. The bill has its own page: what a run actually costs.