Stationary Distributions and Convergence

A stationary distribution is a probability law that a Markov chain preserves under its dynamics; under broad conditions the chain forgets its starting point and converges to this equilibrium.

Definition

A stationary distribution is a probability vector left invariant by the transition matrix, so that a chain started in it remains distributed according to it at every later time; convergence theory studies when and how fast an arbitrary initial distribution approaches this equilibrium.

Scope

This topic covers invariant and stationary distributions and their characterisation as left eigenvectors of the transition matrix, existence and uniqueness criteria, detailed balance and reversibility, the convergence theorem for irreducible aperiodic chains, total-variation distance and mixing times, and coupling and spectral methods for bounding the rate of convergence.

Core questions

• What is a stationary distribution and how is it computed from the transition matrix?
• Under what conditions is the stationary distribution unique and the limit of the chain?
• What does reversibility add, and how is it linked to detailed balance?
• How is the speed of convergence to equilibrium quantified and bounded?

Key theories

Convergence-to-equilibrium theorem

For an irreducible, aperiodic, positive-recurrent chain the distribution after n steps converges to the unique stationary distribution from any starting point, so the chain asymptotically loses memory of its origin.

Reversibility and detailed balance

A chain satisfying the detailed balance equations with respect to a distribution is reversible and has that distribution as stationary; reversibility yields self-adjoint transition operators and underlies spectral bounds on mixing.

Clinical relevance

Stationary distributions describe the long-run fraction of time a system spends in each state, giving steady-state queue lengths, equilibrium frequencies in genetics, and the target laws sampled by Markov chain Monte Carlo; mixing-time bounds determine how long such simulations must run to produce reliable samples.

History

Doeblin and Kolmogorov established the convergence theory in the 1930s using coupling and analytic arguments. The quantitative study of mixing times, sharpened by Diaconis and collaborators from the 1980s, connected convergence rates to the spectral gap and to phenomena such as the cutoff in total-variation distance.

Key figures

• Wolfgang Doeblin
• Andrey Kolmogorov
• Persi Diaconis

Seminal works

• levinPeres2017

Frequently asked questions

How do you find a chain's stationary distribution?

Solve for the probability vector that is left unchanged when multiplied by the transition matrix; for reversible chains the detailed balance equations often give it more directly.

What is a mixing time?

It is the number of steps after which the chain's distribution is within a small total-variation distance of its stationary distribution, measuring how quickly the chain reaches equilibrium.