Metropolis-Hastings Algorithm
The Metropolis-Hastings algorithm builds a Markov chain targeting any posterior by proposing moves and accepting them with a probability that enforces detailed balance.
Definition
The Metropolis-Hastings algorithm generates a Markov chain by drawing a candidate from a proposal distribution and accepting it with probability equal to the minimum of one and the ratio of target densities times the ratio of proposal densities, which guarantees the posterior as the stationary distribution.
Scope
This topic covers the proposal-and-acceptance mechanism, the acceptance ratio that corrects for proposal asymmetry, special cases such as random-walk and independence samplers, and the tuning of proposal scale to achieve efficient mixing.
Core questions
- How does the acceptance probability enforce detailed balance with respect to the target?
- How do random-walk and independence proposals differ in behavior?
- How is the proposal scale tuned, and what acceptance rate is efficient?
- Why does the algorithm need only the unnormalized posterior density?
Key concepts
- proposal distribution
- acceptance probability
- Hastings ratio
- random-walk Metropolis
- independence sampler
- detailed balance
- proposal tuning
Key theories
- Metropolis-Hastings acceptance rule
- Accepting proposals with the Hastings ratio makes the chain reversible with respect to the target, so it converges to the posterior regardless of the proposal, provided the chain is irreducible and aperiodic.
- Optimal scaling
- For random-walk proposals in high dimension, tuning the step size toward an acceptance rate near one quarter balances exploration against rejection, a result from the diffusion-limit analysis of the sampler.
Clinical relevance
Metropolis-Hastings is the general-purpose engine for sampling posteriors in models without conjugate structure, used across statistical genetics, image analysis, and the physical sciences.
History
The algorithm was introduced for statistical physics simulations by Metropolis and the Rosenbluths and Tellers in 1953; Hastings generalized it to arbitrary proposals and statistical targets in 1970, after which it became the cornerstone of MCMC.
Key figures
- Nicholas Metropolis
- Marshall Rosenbluth
- Arianna Rosenbluth
- W. Keith Hastings
Related topics
Seminal works
- metropolis1953
- hastings1970
Frequently asked questions
- What acceptance rate should I aim for?
- For high-dimensional random-walk proposals an acceptance rate around 20-25% is often near optimal, while for one-dimensional or independence proposals higher rates can be appropriate; the goal is efficient exploration, not a particular rate per se.