Metropolis-Hastings Algorithm
Metropolis-Hastings Markov Chain Monte Carlo Algorithm · Also known as: MH algorithm, M-H algorithm, Metropolis algorithm, Metropolis-Hastings sampler, acceptance-rejection MCMC, general-purpose MCMC sampler
The Metropolis-Hastings (MH) algorithm is a general-purpose Markov chain Monte Carlo (MCMC) method for drawing samples from any probability distribution whose density can be evaluated up to a normalising constant. Introduced by Metropolis, Rosenbluth, Rosenbluth, Teller, and Teller (1953) in computational physics and generalised by Hastings (1970) to asymmetric proposal distributions, it is the foundational algorithm from which nearly all subsequent MCMC samplers — Gibbs sampling, Hamiltonian Monte Carlo, slice sampling — are derived or can be viewed as special cases.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+2 more
When to use it
Use the Metropolis-Hastings algorithm whenever the target posterior distribution cannot be sampled from directly and no conjugate or closed-form solution is available. It is applicable to virtually any Bayesian model — regression, hierarchical models, latent variable models, mixture models — as long as the unnormalised posterior density can be evaluated pointwise. Key assumptions are that the target distribution has the same support as the proposal (irreducibility), that the chain is aperiodic, and that sufficient iterations are run for convergence. It becomes inefficient in very high-dimensional parameter spaces, where Hamiltonian Monte Carlo is typically preferred.
Strengths & limitations
- Completely general: applies to any target distribution evaluable up to a constant, with no conjugacy or closed-form requirements.
- The normalising constant of the posterior (the marginal likelihood) is never required — it cancels in the acceptance ratio.
- Asymptotically correct: the stationary distribution of the chain is exactly the target, guaranteed by detailed balance.
- Conceptually transparent and straightforward to implement for low- to moderate-dimensional problems.
- Serves as the backbone for more advanced samplers (Gibbs, HMC, NUTS, reversible-jump MCMC) which are all special or extended cases.
- Efficiency degrades sharply with dimension: tuning the proposal scale to achieve the optimal acceptance rate (around 23% for a random-walk Gaussian proposal in high dimensions) becomes progressively harder.
- Correlated samples — inherent to any Markov chain — inflate Monte Carlo standard errors relative to independent samples; effective sample size is always less than the raw chain length.
- Convergence is asymptotic and must be diagnosed empirically (R-hat, effective sample size, trace plots); there is no finite-time guarantee of proximity to the target.
- Mixing can be extremely slow when the posterior is multimodal, banana-shaped, or has strong parameter correlations.
Frequently asked
How do I choose the proposal distribution?
For a random-walk proposal, the standard recommendation is to tune the proposal standard deviation so that the acceptance rate falls between roughly 20% and 50%; for a d-dimensional Gaussian target the theoretically optimal rate is about 23.4%. In practice, run short pilot chains, adjust the scale, and verify that trace plots show healthy mixing. Adaptive MH methods automate this tuning by updating the proposal covariance online during burn-in.
What is the difference between the Metropolis and the Metropolis-Hastings algorithm?
The original 1953 Metropolis algorithm requires a symmetric proposal distribution, so the proposal ratio q(θ^(t)|θ*)/q(θ*|θ^(t)) equals 1 and drops from the acceptance ratio. Hastings' 1970 generalisation retains this correction factor, enabling any proposal distribution — including directed, independent, or data-driven proposals — while preserving the correct stationary distribution.
How do I know when the chain has converged?
Run at least four chains from overdispersed starting points. Compute the Gelman-Rubin potential scale reduction factor R-hat for each parameter; values below 1.01 indicate acceptable convergence. Also inspect trace plots for stationarity and compute the effective sample size (ESS); a minimum ESS of around 400 per parameter is a common practical threshold. A bulk ESS and tail ESS are reported by modern software such as ArviZ and Stan.
When should I prefer Hamiltonian Monte Carlo over Metropolis-Hastings?
Hamiltonian Monte Carlo (HMC) exploits gradient information about the posterior to make large, informed moves with high acceptance rates. For smooth, continuous posteriors in moderate to high dimensions (roughly 10 or more parameters), HMC and its adaptive variant NUTS substantially outperform a random-walk MH sampler in effective samples per second. Plain MH remains useful for low-dimensional problems, discrete parameter spaces (where gradients are unavailable), and as a pedagogical and theoretical reference point.
Sources
- Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953). Equation of state calculations by fast computing machines. The Journal of Chemical Physics, 21(6), 1087–1092. DOI: 10.1063/1.1699114 ↗
- Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1), 97–109. DOI: 10.1093/biomet/57.1.97 ↗
- Robert, C. P., & Casella, G. (2004). Monte Carlo Statistical Methods (2nd ed.). Springer. ISBN: 978-0-387-21239-5
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian Data Analysis (3rd ed.). CRC Press. ISBN: 978-1-439-84095-5
How to cite this page
ScholarGate. (2026, June 3). Metropolis-Hastings Markov Chain Monte Carlo Algorithm. ScholarGate. https://scholargate.app/en/bayesian/metropolis-hastings-algorithm
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian RegressionBayesian↔ compare
- Gibbs SamplingBayesian↔ compare
- Hamiltonian Monte CarloBayesian↔ compare
- Sequential Monte CarloBayesian↔ compare
- Slice SamplingBayesian↔ compare