Neural Mastery
← Practice
A/B Test Statistical Significance (Two-Proportion Z-Test)hard

A/B Test Statistical Significance (Two-Proportion Z-Test)

hard ML Systems & MLOps 15 min

NumPy allowed · plain Python earns 10 bonus points

Implement the real two-proportion z-test statistic used to check whether an A/B test's conversion-rate difference is statistically significant, not just noise.

Task

Implement ab_test_z_score(conversions_a, visitors_a, conversions_b, visitors_b). Compute p_a = conversions_a/visitors_a, p_b = conversions_b/visitors_b, pooled proportion p_pool = (conversions_a+conversions_b)/(visitors_a+visitors_b), and standard error se = sqrt(p_pool*(1-p_pool)*(1/visitors_a + 1/visitors_b)). Return the z-score (p_b - p_a) / se. Return 0.0 if se is 0.

Function Signature

ab_test_z_score(conversions_a: int, visitors_a: int, conversions_b: int, visitors_b: int) -> float

Examples

Example 1: clear improvement
Input: conversions_a = 100 visitors_a = 1000 conversions_b = 150 visitors_b = 1000
Output: 3.3806170189140654
Example 2: no difference
Input: conversions_a = 100 visitors_a = 1000 conversions_b = 100 visitors_b = 1000
Output: 0
Example 3: b performs worse
Input: conversions_a = 200 visitors_a = 1000 conversions_b = 100 visitors_b = 1000
Output: -6.262242910851494

Constraints

  • visitors_a, visitors_b > 0
Python3Saved ✓
import math

def ab_test_z_score(conversions_a, visitors_a, conversions_b, visitors_b):
# Your implementation here
pass

Case 1: clear improvement
Input: conversions_a = 100 visitors_a = 1000 conversions_b = 150 visitors_b = 1000
Expected: 3.3806170189140654
Case 2: no difference
Input: conversions_a = 100 visitors_a = 1000 conversions_b = 100 visitors_b = 1000
Expected: 0
Case 3: b performs worse
Input: conversions_a = 200 visitors_a = 1000 conversions_b = 100 visitors_b = 1000
Expected: -6.262242910851494