5 pieces · 52 minutes
Evaluation and safety
How you find out whether it worked, and how you keep knowing after it ships.
Covers: Agent evaluation · Evaluation & safety
Anthropic's new frontier-lab measurements make agent oversight more visible. The important engineering lesson is that activity metrics, control metrics, and outcome metrics answer different questions—and none can replace the others.
02A Managed Agent Harness Does Not Remove Your Evidence ProblemEvaluation & safety39mOpenAI's Agents API manages the Codex harness for long-running work. That changes who operates the runtime—not the evidence an engineering team needs before it can trust an agent's result.
03Agent Evaluations Are Security SystemsEvaluation & safety511mA realistic agent evaluation needs more than a benchmark and a score. It needs explicit scope, least-privileged tools, observable actions, stop conditions, and a recovery plan.
04AI Agent Evals: How Do You Know an Agent Actually Works?Agent evaluation212mA convincing final message is not proof of completion. Agent evaluations check the task, the actual environment, repeated trials, and the evidence left behind.
05Why an Agent Eval Can Pass on Monday and Fail on FridayAgent evaluation211mA passing eval is evidence about one version of an agent system. When the model, context, tools, policies, or environment change, the evidence needs to be renewed.