Compute Eval Confidence Interval (Wilson Score)hard

Compute Eval Confidence Interval (Wilson Score)

hard Agents in Production 15 min

NumPy allowed · plain Python earns 10 bonus points

Implement the Wilson score confidence interval, a real, more statistically sound alternative to a naive normal-approximation interval for reporting uncertainty on a small-sample eval pass rate.

Task

Implement wilson_interval(successes, total, z) using the real Wilson score formula: p = successes/total; denom = 1 + z**2/total; center = (p + z**2/(2*total)) / denom; margin = (z * sqrt(p*(1-p)/total + z**2/(4*total**2))) / denom. Return (center - margin, center + margin).

Function Signature

wilson_interval(successes: int, total: int, z: float) -> tuple

Examples

Example 1: large sample tight interval
Input: successes = 850
total = 1000
z = 1.96
Output: [0.8265308859716873, 0.8707902850178393]

Example 2: small sample wide interval
Input: successes = 8
total = 10
z = 1.96
Output: [0.49015684672072335, 0.9433190520193067]

Example 3: perfect score still has real uncertainty
Input: successes = 10
total = 10
z = 1.96
Output: [0.7224598312333834, 1]

Constraints

total > 0
z > 0 (e.g. 1.96 for ~95% confidence)

Python3Saved ✓

import math

def wilson_interval(successes, total, z):

# Your implementation here

pass

Case 1: large sample tight interval

Input: successes = 850
total = 1000
z = 1.96

Expected: [0.8265308859716873, 0.8707902850178393]

Case 2: small sample wide interval

Input: successes = 8
total = 10
z = 1.96

Expected: [0.49015684672072335, 0.9433190520193067]

Case 3: perfect score still has real uncertainty

Input: successes = 10
total = 10
z = 1.96

Expected: [0.7224598312333834, 1]