Evaluating AgentsOpen in Colab
Agentic project loop
Task definition·Context & tools·Orchestration loop·Evaluation·Guardrails·Operations feedback

Evaluating Agents

A model produces one output you can score against a reference. An agent produces a trajectory (a sequence of thoughts, tool calls, and observations) that may reach the right answer by a wasteful or unsafe path, or reach a wrong answer through reasoning that looked fine at every step. Evaluating agents means scoring both what they produced and how they got there.

Real-world examples: measuring whether a coding agent's patch passes the hidden test suite (SWE-bench); checking whether a web agent completed a multi-step task without hallucinating a step (GAIA); auditing whether a support agent followed policy before issuing a refund (τ-bench).

Outcome vs trajectory

Two complementary axes of evaluation:

  • Outcome (end-state) evaluation: did the final result satisfy the goal? Often checkable programmatically: tests pass, the order shows refunded, the file contains the value. This is the metric users care about.
  • Trajectory (process) evaluation: was the path sound? Did the agent call the right tools, avoid forbidden actions, not loop, and stay within budget? Two agents with identical success rates can have wildly different trajectories — one efficient and auditable, the other a 40-step random walk that got lucky.

A useful efficiency metric normalises steps taken against an oracle minimum:

efficiency=minimum stepssteps taken,success rate=1N∑i=1N1[task i solved]\text{efficiency} = \frac{\text{minimum steps}}{\text{steps taken}}, \qquad \text{success rate} = \frac{1}{N}\sum_{i=1}^{N} \mathbb{1}[\text{task } i \text{ solved}]

Pass@k hides flakiness

Agents are stochastic, so a single run is noise. Report pass@k (solved in at least one of k attempts) and pass^k / consistency (solved in all k). A high pass@1 with low consistency means the agent is unreliable, not capable — exactly the distinction a single-run benchmark conceals.

LLM-as-judge on traces

Many trajectory questions have no programmatic checker — "did the agent's reasoning justify the action?", "was the tone appropriate?". Here an LLM-as-judge scores the trace against a rubric. Make it reliable by:

  • Using a rubric with explicit criteria rather than a vague "is this good?".
  • Preferring pairwise comparison (A vs B) over absolute scores — models are better at ranking than calibrated grading.
  • Mitigating bias: judges favour longer answers, their own outputs, and the first option presented. Randomise order and control for length.

Validate the judge before you trust it

An LLM judge is itself a model under evaluation. Spot-check it against human labels on a held-out set and report its agreement (e.g. Cohen's κ) before using it as ground truth. An unvalidated judge can silently bless a regressed agent.

Benchmarks and budgets

Public agent benchmarks pin down capability claims: GAIA (multi-step reasoning + tool use), SWE-bench (resolve real GitHub issues), WebArena (browser tasks), τ-bench (tool-agent-user interaction with policy). They are necessary but not sufficient — your task distribution differs, and benchmarks leak into training data over time.

Two budgets gate every production agent and belong in the eval report:

  • Cost: tokens × steps × price. An agent that solves a task for $3 may be a worse product than a workflow that solves 90% of them for $0.02.
  • Latency: steps are sequential, so end-to-end time grows with trajectory length. Parallelising independent tool calls is often the highest-leverage fix.

For the full picture — how these metrics organise into four levels (single-agent correctness, per-agent profiling, multi-agent trajectory correctness, and whole-system health), the latency metrics that decide usability, and the leading indicators that predict an effective agent — see the deep dive:

Every piece above is easy to state and hard to size. Build the eval stage by stage below, on two simulated agents with known true skills: success rate with the interval it actually carries, trajectory metrics, pass@k against consistency, a judge scored against human labels, then budgets. The tasks and runs per task controls re-run the whole eval, which is the point, because most of what an eval reports is a function of how many times you looked.

Building an agent eval, step by step

Measuring the agentstep 1 of 6
One trajectory is an anecdote

Here is a single run: a sequence of tool calls ending in success or failure. It is the unit everything else is built from, and on its own it tells you almost nothing — an agent is stochastic, so this is one draw from a distribution you have not measured.

It does contain more than a verdict, though. The path is visible: which tools were called, how many steps it took against the 5 an oracle needs, and whether it touched anything it should not have. Outcome and trajectory are both in here; the rest of the walkthrough is about extracting them at a scale where they mean something.

Two agents, two trajectories, same task. Note that the slower one is not obviously the worse one yet.

tasksruns per task
One run50 tasks · k = 3
Agent A · task 1 · 11 stepsplansearchreadsearchreadcomparereaddraftverifysubmit+1 moreoutcome: failedefficiency 5/11 = 0.45policy violations: 0Agent B · task 1 · 16 stepsplansearchreadsearchreadcomparereaddraftverifysubmit+6 moreoutcome: solvedefficiency 5/16 = 0.31policy violations: 0one draw each · everything below is about doing this enough times to mean something
What the eval can claim so far
sample

50 tasks × 3 runs = 150 trajectories per agent.

Agent AAgent B

Two simulated agents with fixed true skills, evaluated the way the lesson describes: outcome, then trajectory, then repeated runs, then a judge that is itself validated, then budgets. Wilson intervals, pass@k and pass^k, Cohen's κ and cost per solved task are all computed from the runs. The task-count and repeat controls re-run the whole eval, which is the point — most of what an eval reports is a function of how many times you looked.

Deep dive
Agentic Analysis: A Metrics Taxonomy
Includes runnable notebook

Exercise

Which question is answered by *trajectory* (process) evaluation rather than *outcome* evaluation?

Need a hint?

Outcome = did the final result satisfy the goal. Trajectory = was the path sound?

Exercise

An agent solves a task in 1 of 5 attempts. Why report pass@k AND consistency rather than a single run?

Need a hint?

Agents are stochastic; one run is noise.

Common mistakes

  1. Scoring only the final answer: missing unsafe or wasteful trajectories that happen to succeed.
  2. Single-run evaluation: reporting one lucky trajectory instead of pass@k with consistency.
  3. Trusting an unvalidated judge: using LLM-as-judge as ground truth without checking it against humans.
  4. Ignoring cost and latency: shipping an agent that is accurate but too slow or expensive to use.

Key takeaways

  • Evaluate outcome (did it work) and trajectory (was the path sound, efficient, safe).
  • Agents are stochastic — report pass@k and consistency, not a single run.
  • LLM-as-judge scores unverifiable trace quality; use rubrics, pairwise comparison, and validate the judge against humans.
  • Public benchmarks bound capability; cost and latency budgets decide whether the agent actually ships.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

Builds onRelatedGo deeper
Evaluating AgentsBuilding with LLMsAttention & TransformersReflection PatternsTool Use, Function Callin…LLM Evaluation & LLM-as-a…Deploying Agents in Produ…Observability for AgentsAgentic Analysis: A Metri…The Agentic Project Loop