evals
We build the eval harness before the agent
Most agent demos work on a happy path and break on edge cases. We build the eval set first, then the agent, so when something regresses you find out in CI, not in production.
The order most teams build agents in is wrong. They build the agent first, get it working on a few demo questions, ship it, and then start writing evals when complaints start rolling in. By that point the prompt has been hand-tuned a dozen times, every regression is a guessing game, and the team is afraid to touch it.
We build the eval harness first. The agent is built against it.
What “first” actually means
On day one, we sit with the team and write the failure modes the agent must never exhibit. These become the first tests:
- “If the retriever returns nothing, the agent does not answer.”
- “If the question is about medical dosing, the agent escalates to a human.”
- “If the question contains ‘as my doctor’, the agent escalates regardless of content.”
- “If a tool call would write to the CRM, the agent confirms before proceeding.”
These rules are the load-bearing structure of the agent. We write tests for each one before any prompt or retriever is built. The tests start red (correct: the agent doesn’t exist yet), and the implementation is finished when they’re green.
Then the happy path
After the failure modes are tested, we add the happy-path questions: representative things the user is actually going to ask. These are sourced from the team: interview the support manager for an hour, get 50 actual questions from the last week’s tickets, code them into the eval set with expected answers.
The agent is then built to pass them.
Why this changes everything
The order matters because:
- The shape of the agent comes from the failure modes, not the happy path. Building happy-path-first gives you an agent that does well on five demo questions and fails on the sixth. Building failure-mode-first gives you an agent whose architecture refuses bad inputs by default.
- Regressions are caught in CI. Every prompt tweak, every model swap, every retriever change runs the full eval set. If a “small improvement” regresses an adversarial test, the deploy is blocked. The team finds out in seconds, not after a customer complaint.
- The team owns the agent’s behaviour. The eval set is the spec. The support manager and compliance lead can read it. They sign off on what the agent will and won’t do. There’s no longer a black box where somebody changed the prompt last week and nobody knows.
The harness is unglamorous. Nobody screenshots the eval dashboard for the investor deck. But it’s the reason the agent is still working in month six instead of being quietly turned off after the third bad answer.