Bayes' Theorem: Posterior From Scratch

Difficulty: Medium · Concept: Probability & Statistics — Bayes' Theorem

The classic framing: a test for a rare condition has some prior probability of the condition (prior), a sensitivity (probability the test is positive given the condition is present), and a false positive rate (probability the test is positive given the condition is absent). Given a positive result, what's the actual probability the condition is present?

$P(\text{condition} \mid \text{positive}) = \frac{P(\text{positive} \mid \text{condition}) \, P(\text{condition})}{P(\text{positive} \mid \text{condition})P(\text{condition}) + P(\text{positive} \mid \neg\text{condition})P(\neg\text{condition})}$

This is the exact mechanism behind why a rare-disease test with a seemingly-impressive sensitivity can still have a low posterior probability of a positive result being real — one of the most consistently counter-intuitive real results in applied probability.

Your task: implement bayes_posterior(prior, sensitivity, false_positive_rate), returning $P(\text{condition} \mid \text{positive})$ .

Implement it yourself

assert abs(bayes_posterior(0.01, 0.99, 0.05) - 1 / 6) < 1e-9
assert abs(bayes_posterior(0.5, 0.8, 0.1) - 8 / 9) < 1e-9
assert abs(bayes_posterior(0.2, 0.9, 0.3) - 3 / 7) < 1e-9
assert abs(bayes_posterior(0.01, 1.0, 0.0) - 1.0) < 1e-9

Next: Numerical Gradient via Finite Differences