Why does the normal distribution appear so often?

Because the central limit theorem makes it the limiting distribution of standardized sums of many independent small effects, so any quantity built up from many comparable contributions tends to be approximately normal regardless of the details.

How are the exponential and Poisson distributions connected?

They describe the same process from two angles: in a Poisson process the number of events in a fixed interval is Poisson distributed while the waiting times between events are exponentially distributed.

Common Probability Distributions

A small catalogue of distribution families, the binomial, Poisson, geometric, uniform, normal, exponential, and gamma among them, recurs across probability and statistics because each arises from a simple and frequently met generating mechanism.

Definition

Common probability distributions are the standard parametric families of laws, each defined by a probability mass or density function with a few parameters, that model the most frequently encountered patterns of randomness and serve as building blocks of probabilistic models.

Scope

The topic covers the principal discrete families such as Bernoulli, binomial, geometric, negative binomial, and Poisson, and the principal continuous families such as uniform, exponential, gamma, beta, and normal, together with their generating mechanisms, moments, and characteristic functions, and the limiting and structural relations that connect them.

Core questions

What generating mechanism gives rise to each standard distribution?
How are the discrete and continuous families related by limits and transformations?
What are the moments and characteristic functions of the standard families?
Why does the normal distribution occupy a central place among them?

Key concepts

Bernoulli and binomial
Poisson and exponential
gamma and beta families
normal distribution
relations among families

Key theories

Poisson limit of the binomial: When the number of independent trials grows while the success probability shrinks so that the expected number of successes stays fixed, the binomial distribution converges to the Poisson, explaining why rare-event counts are Poisson distributed.
Normal distribution as a universal limit: The normal distribution arises as the limiting law of standardized sums of many small independent contributions, which is why it models measurement error and aggregate quantities and serves as the reference distribution of classical statistics.

Clinical relevance

These families are the default models throughout applied probability and statistics: the Poisson and exponential describe arrivals and lifetimes in reliability and queueing, the binomial and its relatives describe counts of successes in trials and surveys, and the normal underpins measurement-error models, confidence intervals, and a great deal of statistical inference.

History

The named distributions accumulated over three centuries: Bernoulli and de Moivre studied counts and the normal approximation, Poisson derived the law of rare events, and Gauss and Laplace established the normal distribution for errors. The modern treatment organizes them by their generating mechanisms and limiting relations.

Key figures

Abraham de Moivre
Simeon Denis Poisson
Carl Friedrich Gauss
Jacob Bernoulli

Seminal works

feller1968

Frequently asked questions

Why does the normal distribution appear so often?: Because the central limit theorem makes it the limiting distribution of standardized sums of many independent small effects, so any quantity built up from many comparable contributions tends to be approximately normal regardless of the details.
How are the exponential and Poisson distributions connected?: They describe the same process from two angles: in a Poisson process the number of events in a fixed interval is Poisson distributed while the waiting times between events are exponentially distributed.