Central Limit Theorem

The central limit theorem says that the sum of many independent random variables, once centered and rescaled, has an approximately normal distribution regardless of the shape of the individual variables, which is why the bell curve appears throughout science.

Definition

The central limit theorem states that for independent identically distributed random variables with finite mean and variance, the standardized sum converges in distribution to the standard normal law as the number of terms grows.

Scope

The topic covers the classical central limit theorem for independent identically distributed variables with finite variance, the Lindeberg and Lyapunov conditions for triangular arrays of independent variables, the characteristic-function method of proof, the Berry-Esseen bound on the rate of convergence, and the extension to non-Gaussian stable limits when variance is infinite.

Core questions

• Why is the normal distribution the universal limit of standardized sums?
• What conditions, such as Lindeberg's, are needed when the summands are not identically distributed?
• How fast does the distribution of a normalized sum approach the normal law?
• What replaces the normal limit when the variance is infinite?

Key concepts

• convergence in distribution
• Lindeberg condition
• Lyapunov condition
• Berry-Esseen rate
• stable limits

Key theories

Classical central limit theorem

For independent identically distributed variables with finite variance, the sum minus its mean and divided by the square root of the number of terms times the standard deviation converges in distribution to the standard normal, proved cleanly by characteristic functions.

Lindeberg-Feller theorem

For triangular arrays of independent variables the Lindeberg condition, that no single term contributes a non-negligible share of the variance, is sufficient and essentially necessary for asymptotic normality, giving the theorem its most general classical form.

Berry-Esseen bound

When a finite third moment exists, the maximum error of the normal approximation to the distribution of a standardized sum is bounded by a constant times the third absolute moment divided by the variance to the three-halves power and the square root of the sample size.

Clinical relevance

The central limit theorem is the cornerstone of statistical inference: it justifies the normal approximation behind confidence intervals, z-tests and t-tests, and the asymptotic distribution of estimators, and it explains why measurement errors and aggregated quantities across the sciences are so often approximately Gaussian.

History

De Moivre and Laplace found the normal approximation to the binomial in the eighteenth century. Lyapunov gave the first rigorous general proof using moments, Lindeberg supplied the definitive condition, and Feller showed it was essentially necessary, while Berry and Esseen quantified the rate of convergence.

Key figures

• Abraham de Moivre
• Pierre-Simon Laplace
• Aleksandr Lyapunov
• Jarl Waldemar Lindeberg

Seminal works

• billingsley1995

Frequently asked questions

Does the central limit theorem require the summands to be normally distributed?

No; the remarkable point is that the individual variables can have almost any distribution with finite variance, and their standardized sum still tends to the normal law as the number of terms grows.

How large must the sample be for the normal approximation to be good?

There is no universal answer; the Berry-Esseen bound shows the error depends on the third moment and decays like one over the square root of the sample size, so skewed or heavy-tailed summands require larger samples for a good approximation.