Bayes' Theorem
Updating beliefs in light of evidence
Bayes' theorem describes how to update the probability of a hypothesis when new evidence arrives. The core logic is that the posterior probability is proportional to the likelihood multiplied by the prior probability. The theorem explains why a positive result on a test for a rare disease can still leave the probability of actually having that disease surprisingly low, a phenomenon known as the base-rate effect. It is the mathematical foundation of Bayesian inference, diagnostic reasoning, and likelihood ratios.
Concept and Formula
Bayes' theorem updates our prior belief about a hypothesis H after observing data D. The formal statement is P(H|D) = P(D|H) × P(H) / P(D). Here P(H|D) is the posterior probability, P(D|H) is the likelihood, and P(H) is the prior. The denominator P(D) is a normalising constant computed as the sum of likelihood times prior across all competing hypotheses. In plain words: even strong evidence in favour of a hypothesis will not raise the posterior much if the prior probability of that hypothesis is very low to begin with.
Computing and Reading: The Base-Rate Effect
A concrete example illustrates the base-rate effect. Suppose 1 in 1000 people in a population has a certain disease, a test has 99 percent sensitivity, and a 5 percent false-positive rate. In a population of 100 000, roughly 100 people are ill; of these, 99 test positive. Of the healthy 99 900, about 4995 also test positive. Among the 5094 total positive results, only 99 are true cases. The probability of actually having the disease given a positive result is therefore roughly 2 percent. This striking result follows directly from applying the formula and shows how ignoring the base rate leads to serious overestimation of disease probability.
Common Misuses and Misconceptions
The most common error is confusing P(H|D) with P(D|H), known as the prosecutor's fallacy. Finding that evidence would be very unlikely if the defendant were innocent does not mean the defendant is very likely guilty, because the prior probability and alternative hypotheses have been ignored. A second frequent mistake is assigning an arbitrary prior and reporting results without testing how sensitive conclusions are to that choice. A third misconception is treating Bayes' theorem as purely a subjective belief system and dismissing it from data-driven applications, when in fact it is equally applicable to objective frequency-based problems.
Why It Matters and How to Report It
Bayes' theorem underlies medical diagnosis, machine learning classifiers, forensic evidence evaluation, and Bayesian model comparison in the social sciences. When a researcher uses Bayesian inference, reporting should include the source and justification of the prior distribution, how the likelihood function was specified, a summary of the posterior distribution with a point estimate and credible interval, and the results of a sensitivity analysis varying the prior. Compared with frequentist p-values, posterior probabilities are directly interpretable as statements such as there is an 87 percent probability that the hypothesis is true, which is one reason Bayesian reporting is increasingly favoured in research literature.