Why an Agent Eval Can Pass on Monday and Fail on Friday

A passing evaluation is a snapshot, not a lifetime guarantee

An evaluation asks a system to complete a defined task, runs one or more trials, and applies grading logic to the output or the resulting environment state. That is valuable evidence. But the result is attached to the system that was actually tested: a particular model, instructions, context assembly process, tool set, task environment, and grader configuration.^[1]

That system can change in quiet ways. A knowledge source is updated. A retrieval ranking shifts. A tool gets a new argument or a different error message. A feature flag changes permissions. A model provider updates a model behind a familiar name. Even a well-intentioned prompt improvement can help one task while weakening another. If the agent is connected to changing systems, a frozen result should be treated as a baseline—not proof that the behavior will remain stable.^[1][2]

Five surfaces that can change agent behavior

1. Model behaviorA new model, model version, decoding setting, or provider-side update can change how the agent interprets instructions, chooses tools, or handles uncertainty.
2. Context and knowledgeDocuments, retrieved results, memory records, examples, and system instructions can become stale, conflict, or arrive in a different order.
3. Tools and permissionsAPIs evolve, schemas change, rate limits appear, tools fail differently, and actions may gain or lose authority. A tool contract is part of agent behavior.
4. Policies and product rulesA change to what requires approval, what can be disclosed, or what counts as completion changes the task the agent is allowed to perform.
5. Environment and graderThe sandbox, test data, reference solution, assertions, and evaluation harness can change too. A moved score may be caused by the measurement system rather than the agent alone.

This does not mean every configuration value deserves the same level of process. It means the team should know which changes are behaviorally meaningful. A spelling correction in an internal comment is not comparable to switching the model, widening a tool’s permission, or replacing the knowledge base the agent relies on.

Anthropic’s guidance distinguishes capability evals from regression evals. Capability evals ask what an agent can do well and often target hard tasks. Regression evals protect behavior that has already met the expected bar. Over time, a task that was once difficult can graduate into the regression suite that is run continuously to catch backsliding.^[1]

Move from one-off testing to continuous assurance

Continuous assurance is a practical operating habit, not a claim that an agent can be made permanently correct. It combines representative regression tasks, targeted capability tasks, production monitoring, and a record of changes. The goal is to notice meaningful behavioral movement before users discover it by accident.^[1]

change: search tool response schema updated
version: agent-support-v14
affected behavior: source selection and citation formatting
checks rerun: research-regression-suite, source-grounding rubric
baseline: v13 outcome, latency, tool calls, review notes
result: 18/20 tasks unchanged; 2 citation failures
decision: hold release; update parser and add both failures to regression suite

The point of this record is not bureaucracy. When a score changes, it gives a team a starting point for diagnosis. Did the outcome change? Did the agent take a different tool path? Did latency or retries increase? Did the grader change? Did the task data stop representing the real workflow? A percentage without this context often produces confident but unhelpful explanations.

When a score moves, compare traces—not only percentages

A pass rate is a useful summary, but it compresses many different behaviors into one number. Two versions can have the same pass rate while one version uses twice as many tool calls, takes longer, needs more retries, or succeeds through a brittle shortcut. Likewise, a lower score may reveal a broken task specification or a changed grader rather than a degraded agent.^[1]

• Outcome: Did the required environmental state still occur?
• Trace: Which tools, arguments, retries, and observations changed?
• Efficiency: Did time, cost, number of turns, or tool calls move materially?
• Safety and policy: Did the agent attempt an action that should require approval or violate a boundary?
• Measurement: Did the task, reference solution, grader, or test environment change at the same time?

This is especially important for agentic systems because the final answer can hide the path. An agent may reach the right result while ignoring a required source, using an unnecessary write action, or retrying until it happens to succeed. Trace-level evidence lets the team decide whether the behavior is genuinely acceptable before treating the outcome as a durable pass.^[1]

A small operating model for a changing agent

1. Name the release surfacesTrack the model, instructions, retrieval or context strategy, tools, permissions, test environment, grader, and key data sources. You do not need a perfect configuration-management platform to begin.
2. Keep a thin regression suiteStart with real tasks that matter. Include tasks where the agent should act and tasks where it should decline, ask, or hand off.
3. Rerun by change riskRun the relevant checks after material changes. A model switch or permission expansion deserves broader evidence than a cosmetic copy change.
4. Compare against a recorded baselineKeep outcomes, traces, costs, latency, and human-review notes so a result can be interpreted rather than merely observed.
5. Turn failures into durable testsWhen a real failure is understood, add a sanitized version to the suite. The best regression sets are often built from the work the system previously got wrong.

The enduring claim is modest: tests do not remove uncertainty, but they make change legible. A continuously evolving agent needs continuously renewed evidence. That is how a team moves from “it seemed fine in the demo” to a system whose behavior can be inspected, compared, and improved over time.^[1]