Evaluation
A/B test
An A/B test randomly assigns traffic between variants to estimate a causal effect.
How it works
Randomization balances other factors in expectation. The analysis needs a predefined outcome and uncertainty estimate.
A concrete example
Illustrative: 10,000 users per variant yield task completion 61.2% for A and 63.0% for B.
Do not confuse it with Offline evaluation
an A/B test measures live user outcomes; offline evaluation uses recorded or synthetic cases.