ML & LLM Testing

Engineering Foundations for ML introduced the general testing taxonomy (unit, integration, data validation, model, API, load). This page goes deeper into two things that page only named: the specific statistical/model testing techniques classical ML needs, and the LLM-specific test types — prompt tests, golden datasets, hallucination/jailbreak/tool-call tests — that don't have a classical-ML equivalent at all.

Classical ML Testing, In More Depth

• Unit tests: individual functions in isolation — a feature-computation function, a custom loss function's math, a data-cleaning transformation. Standard software unit testing, applied to ML code specifically.
• Data tests: assert incoming data matches expected schema, types, and ranges — see Data Engineering & Versioning; this is the automated, CI-enforced version of that page's validation stage, not a separate concern.
• Schema tests: a stricter, more structural version of data tests — the shape of the data (column names, types, nullability) matches a defined contract exactly, catching an upstream schema change (a renamed column, a type change) before it silently breaks a downstream pipeline stage that assumed the old schema.

• Statistical tests: assert properties of a distribution, not a single value — does a feature's distribution in a new data batch still resemble the training distribution (directly the drift detection math — PSI, KS test — run as an automated test rather than a dashboard you have to remember to check), does a model's prediction distribution look sane (not collapsed to a single class, not wildly different from a known-good baseline run).
• Model tests: beyond accuracy — does the model handle an empty or malformed input gracefully rather than crashing? Does it produce deterministic output for the same input when determinism is expected (see Security & Reproducibility)? Does a known, specific edge case behave as expected — a unit test for model behavior on a case you know matters, not just an aggregate metric.
• Regression tests: run the model against a fixed benchmark dataset and assert its metrics haven't dropped below a threshold — see CI/CD & ML CI/CD for wiring this into an automated deployment gate.
• Performance tests: does the model/pipeline complete within an acceptable time and resource budget — a slow-but-correct model can still fail a production requirement, which accuracy-focused testing alone won't catch.

The regression-test gate as a real, CI-runnable pytest function -- this is what evaluate.py from the CI/CD pipeline actually asserts:

def test_candidate_beats_production_benchmark():
    candidate_f1 = evaluate(candidate_model, benchmark_dataset)
    production_f1 = evaluate(load_production_model(), benchmark_dataset)
    assert candidate_f1 >= production_f1 - 0.01   # allow a small tolerance, not strict improvement every time

LLM-Specific Testing

None of the categories above naturally cover "does this prompt still produce the intended behavior" or "can this model be manipulated into ignoring its instructions" — LLM systems need their own test types:

• Prompt tests: treat a prompt template as code under test — given a fixed set of inputs, does the resulting output still meet expected criteria (format, content, tone)? A prompt change is a code change, and deserves the same regression-testing discipline as any other code change, not ad hoc manual spot-checking.

@pytest.mark.parametrize("query", GOLDEN_DATASET)
def test_support_prompt_stays_on_format(query):
    response = llm.invoke(SUPPORT_PROMPT.format(query=query))
    assert response.startswith(("I can help", "I'm sorry"))   # the two sanctioned opening patterns
    assert "as an AI language model" not in response.lower()  # a known regression this prompt version fixed

• Golden datasets: covered in depth in AI Evaluation — Golden Datasets — the curated, maintained reference set every regression test and prompt test above actually runs against.
• Hallucination tests: specifically check whether the model states claims unsupported by its provided context — see RAG — Evaluating RAG's faithfulness/groundedness metrics, run as an automated test suite rather than only a production-monitoring signal.
• Jailbreak tests: run known jailbreak patterns (see AI Security — Jailbreaks) against the system as part of the regular test suite, not just a one-time pre-launch red-teaming exercise — catching a regression where a previously-patched jailbreak starts working again after a model or prompt update.

• Tool-call tests: for agentic systems, assert that a given input reliably produces the correct tool call with correctly-formed arguments — directly testing the tool-call accuracy dimension of agent evaluation as an automated, CI-enforced check rather than only a periodic manual evaluation.
• Structured-output tests: when a model is expected to return output conforming to a schema (JSON with specific fields, a specific format), assert the output actually validates against that schema — a narrower, more mechanical check than general output-quality evaluation, but one that catches a real and common failure mode (a model drifting from the requested format) cheaply and reliably.

from pydantic import BaseModel

class TicketClassification(BaseModel):
    category: str
    priority: int
    summary: str

def test_ticket_classification_matches_schema():
    raw = llm.invoke(CLASSIFY_PROMPT.format(ticket=SAMPLE_TICKET))
    TicketClassification.model_validate_json(raw)  # raises if the model drifted from the requested schema

• Agent trajectory tests: for multi-step agents, assert not just the final outcome but that the sequence of steps taken matches an expected pattern for known test scenarios — the automated-test version of the trajectory analysis evaluation technique, run against a fixed set of scripted scenarios in CI.

Wiring This Into CI/CD

None of the above matters if it doesn't actually run automatically and block a bad deploy — see CI/CD & ML CI/CD for how model/prompt regression tests, drift checks, and the LLM-specific tests above become an automated gate a change has to pass before shipping, the same discipline as any other software test suite, just testing statistical and generative properties instead of only deterministic logic.

Next: Production Reliability — what happens after tests pass and the system is actually serving real traffic.