LLM, RAG & Agent Evaluation

Three progressively harder evaluation problems — a single text output, a text output grounded in retrieved context, and a multi-step sequence of decisions and tool calls — each needing a different evaluation approach because "correct" means something structurally different for each.

Intuition: Every Evaluator Is Itself Something to Evaluate

The theme connecting every section below: the thing doing the measuring (an LLM-judge, a coarse success metric) has its own failure modes, and trusting it without checking those failure modes just relocates the "is this measurement trustworthy" problem from Evaluation Fundamentals one level down instead of solving it. An LLM-judge can be biased by irrelevant surface features. A single success/fail metric can hide exactly how an agent got there. Both are demonstrated below with real, computed numbers.

LLM Evaluation and LLM-as-a-Judge, In Depth

LLM Evaluation & RAGOps covers the tooling (Ragas, DeepEval, LangSmith, Arize Phoenix) and the traditional-metrics-vs-LLM-as-judge distinction. Going one level deeper into how an LLM-as-judge actually works, and where it breaks.

The basic mechanism: prompt a (typically strong, often larger) LLM with the input, the output being evaluated, and explicit grading criteria, asking it to produce a score or judgment — effectively using a model's own language understanding as the measurement instrument, which scales far better than human review while capturing semantic quality that n-gram metrics (BLEU/ROUGE) structurally cannot.

Position Bias, Made to Flip a Verdict

In pairwise comparisons, judges tend to favor whichever response is shown first. Toggle the display order below on the exact same two responses and watch the verdict actually flip:

Standard mitigation: evaluate both orderings and check for consistency — precisely what the toggle above simulates.

Verbosity Bias, Measured With a Real Correlation

Judges tend to rate longer responses as better, independent of actual quality. Real quality is generated independent of length below; watch a real Pearson correlation appear anyway as judge bias increases:

Mitigated by explicit rubric criteria that don't reward length — not by hoping the judge doesn't notice length at all.

Self-preference bias rounds out the three: a model tends to rate its own outputs more favorably than a different model's — a real reason to use a different, ideally stronger, model as judge rather than a model evaluating itself.

Rubric Design and Validating the Judge

A vague instruction ("rate this response 1-10") produces noisy, low-agreement scores; an explicit rubric (specific criteria, each with concrete examples of what a low/medium/high score looks like) produces far more consistent, defensible judgments. Before trusting an LLM-judge's scores at scale, check its agreement against human judgments on a sample — real agreement, computed, for two rubric styles on the same items:

An unvalidated judge is exactly the "untrustworthy measurement" problem Evaluation Fundamentals warns about, just one layer removed.

RAG Evaluation

Covered in full in RAG — Evaluating RAG: faithfulness/groundedness, answer relevance, context precision/recall, and MRR/NDCG for retrieval ranking quality. The key framing worth repeating here: a RAG system's failure could be in retrieval (wrong or missing chunks), ranking (relevant chunks retrieved but not surfaced near the top), or generation (right context, but the model still didn't use it faithfully) — evaluating each stage separately, not just end-to-end answer quality, is what actually tells you which stage to fix.

Agent Evaluation

Evaluating an agent is structurally different from evaluating a single LLM call — the object being evaluated is a sequence of decisions and actions, not one output, and "did it get the right final answer" can be true even when the path there was unreliable. Click through a real toy trajectory below and watch task success and trajectory quality diverge:

• Task success rate: did the agent actually accomplish the end goal — the most important single metric, but a coarse one that says nothing about how it got there or how reliably it would succeed on a slightly different version of the same task.
• Trajectory analysis: evaluating the sequence of actions the agent took, not just the final outcome — did it use tools in a sensible order, did it recover gracefully from a failed tool call, did it waste steps on unproductive exploration, exactly like step 2 above? A trajectory can be evaluated by a human, by an LLM-judge given the full action log (extending the LLM-as-judge approach above to a sequence rather than a single output), or against a reference trajectory when tasks are constrained enough to have one.
• Tool-call accuracy: specifically, did the agent call the right tool, with correctly-formed and correctly-valued arguments, at each step — a narrower, more mechanical metric than overall trajectory quality, useful for isolating whether failures come from reasoning (choosing the wrong approach) or execution (choosing the right approach but calling the tool incorrectly).
• Efficiency metrics: number of steps/tool calls, total tokens used, and wall-clock latency to complete the task — two agents with identical success rates can have very different production cost/latency profiles, and efficiency is invisible if success rate is the only metric tracked (see AI Cost Engineering for the broader cost-optimization context this feeds into).
• Why this had no dedicated treatment before: single-turn LLM evaluation and RAG evaluation both inherited well-established metric traditions (classification metrics, information-retrieval metrics) to build on. Agent evaluation doesn't have an equally mature inherited tradition — it's a genuinely newer problem, closer to evaluating a multi-step plan or a reinforcement learning policy's rollout than to evaluating a single prediction, and the field's evaluation practice here is still actively maturing.

Regression Testing for Models and Prompts

Extends ML Workflow Fundamentals' classical regression-testing idea (does a change degrade performance against a fixed benchmark) to prompts and models specifically: every prompt change, model version bump, or fine-tune should run against the golden dataset (see Evaluation Fundamentals) before shipping, with a defined threshold for what counts as an acceptable regression — the same automated gate CI/CD & ML CI/CD describes for classical ML, applied to generative and agentic systems where "did this get worse" is measured by the evaluation methods on this page rather than a single accuracy number.

Code: A Real Judge-Bias Smoke Test

The same check the position/verbosity diagrams above simulate, runnable against a real judge model:

import numpy as np
from scipy.stats import pearsonr

# Position bias: run every pairwise comparison in both orders, flag flips.
def check_position_bias(judge_fn, pairs):
    flips = sum(1 for a, b in pairs if judge_fn(a, b) != judge_fn(b, a))
    return flips / len(pairs)  # real flip rate -- should be ~0 for an unbiased judge

# Verbosity bias: correlate judge score against response length directly.
lengths = [len(r.split()) for r in responses]
scores = [judge_fn(r) for r in responses]
r, p_value = pearsonr(lengths, scores)
if abs(r) > 0.3 and p_value < 0.01:
    print(f"Judge score correlates with length (r={r:.2f}) -- likely verbosity bias.")

Next: Human & Adversarial Evaluation — where automated evaluation isn't enough on its own.