Metropolis-Hastings with Missing Data
Also known as: MH with missing data, Metropolis-Hastings data augmentation, MCMC missing data imputation, MH data-augmentation sampler
Metropolis-Hastings with missing data treats unobserved values as latent variables and samples them jointly with model parameters inside a single MCMC chain. By augmenting the target distribution to include both parameters and missing values, the algorithm yields properly calibrated posterior inference without discarding incomplete cases or requiring a separate imputation step.
Key highlights
- Produces a single, coherent posterior that simultaneously estimates parameters and propagates missing-data uncertainty.
- Requires no ad hoc imputation step before the main analysis; imputation is integrated into the MCMC loop.
- Applicable to virtually any parametric model, including non-conjugate and hierarchical models, because only the likelihood ratio is needed.
- Correctly handles data missing at random (MAR) and, with an explicit missingness model, missing not at random (MNAR).
- Posterior predictive checks and model comparison (WAIC, LOO-CV) remain straightforward because full posteriors are available.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use this method when your Bayesian model has missing observations and you want inference that fully accounts for missingness uncertainty rather than discarding incomplete cases (complete-case analysis) or plugging in a single imputed value. It is especially appropriate when data are missing at random (MAR) or when the missingness mechanism is modelled explicitly (MNAR). It is less suitable when missingness is negligible, when the dataset is very large and a fast approximate method suffices, or when a closed-form Gibbs update is available for all blocks — in that case pure Gibbs sampling is simpler and more efficient.
Strengths & limitations
- Produces a single, coherent posterior that simultaneously estimates parameters and propagates missing-data uncertainty.
- Requires no ad hoc imputation step before the main analysis; imputation is integrated into the MCMC loop.
- Applicable to virtually any parametric model, including non-conjugate and hierarchical models, because only the likelihood ratio is needed.
- Correctly handles data missing at random (MAR) and, with an explicit missingness model, missing not at random (MNAR).
- Posterior predictive checks and model comparison (WAIC, LOO-CV) remain straightforward because full posteriors are available.
- Computationally expensive: each iteration must impute all missing values as well as update parameters, which is costly with many missing cells.
- Proposal tuning is critical; a poorly scaled random-walk proposal mixes slowly, especially in high-dimensional missing-data problems.
- Convergence diagnostics (R-hat, effective sample size) must be checked for both parameter and imputation chains.
- When the fraction of missing data is high the posterior can be dominated by the prior, making prior sensitivity analysis essential.
- MNAR models require a correctly specified missingness mechanism, which is usually unverifiable from the observed data alone.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How is this different from multiple imputation followed by pooled analysis?
Multiple imputation (MI) separates imputation from analysis: m imputed datasets are created, analysed independently, and results are pooled with Rubin's rules. Metropolis-Hastings with data augmentation integrates both steps into a single MCMC chain, which is more coherent for complex or hierarchical models where the imputation and analysis models must be fully compatible. For simpler models the two approaches give similar results.
Do I need to model the missingness mechanism explicitly?
Only if data are missing not at random (MNAR). If missingness depends only on observed variables (MAR), the mechanism is ignorable and you need only model the complete-data likelihood. For MNAR you must specify and fit a joint model for data and missingness, which is rarely identifiable without strong assumptions.
How do I diagnose convergence when missing values are also being sampled?
Run multiple chains from dispersed starting values and compute R-hat and effective sample size (ESS) for both model parameters and a summary of imputed values (e.g. the fraction missing or imputed means). Values of R-hat below 1.01 and ESS above a few hundred for all monitored quantities are reassuring.
Can Stan or JAGS handle this automatically?
JAGS handles missing data natively by treating NA nodes as additional unknowns sampled within the MCMC loop. Stan requires explicit declaration of missing-data parameters and an augmented likelihood. Both approaches implement the same statistical principle: data augmentation inside MCMC.
What acceptance rate should I target?
For a random-walk Metropolis-Hastings proposal on a continuous parameter block, theoretical results suggest targeting roughly 23% acceptance in high dimensions and up to 44% for a single scalar parameter. Adaptive MCMC schemes adjust the proposal covariance during warm-up to hit these targets automatically.
Sources
- 1.Tanner, M. A. & Wong, W. H. (1987). The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82(398), 528-540.
- 2.Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A. & Rubin, D. B. (2013). Bayesian Data Analysis (3rd ed.). CRC Press.ISBN 978-1439840955
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Metropolis-Hastings with Missing Data. ScholarGate. https://scholargate.app/bayesian/metropolis-hastings-with-missing-data