Hamiltonian Monte Carlo with Missing Data
Hamiltonian Monte Carlo with Missing Data Imputation · Also known as: HMC with missing data, HMC data augmentation, Bayesian HMC imputation, HMC with data augmentation
Hamiltonian Monte Carlo with missing data extends the gradient-based HMC sampler to handle incomplete observations by treating missing values as additional unknown parameters. The posterior over model parameters and missing values is sampled jointly in one efficient pass, exploiting gradient information to explore the high-dimensional joint space with far fewer rejected proposals than random-walk MCMC.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use HMC with missing data when the analysis involves continuous or mixed-type outcomes with moderate to high rates of missingness (say 5–40%) under a MAR or MCAR assumption, and when the full Bayesian model has smooth, differentiable log-posteriors — such as hierarchical regression, latent-variable, or factor models — where gradient information is available. The method is especially valuable when missingness correlates with model covariates in complex ways that ad hoc complete-case or single-imputation approaches would distort. Prefer simpler Gibbs samplers or Metropolis–Hastings when the likelihood is non-differentiable (e.g. count data with custom link functions without automatic differentiation), or when missingness is MNAR and the mechanism cannot be modelled. Do not use this method when missingness is extremely sparse (under 2–3%) and complete-case analysis introduces negligible bias.
Strengths & limitations
- Jointly samples model parameters and missing values, propagating imputation uncertainty into all downstream estimates without a separate imputation step.
- Gradient-driven proposals explore the high-dimensional joint posterior far more efficiently than random-walk alternatives, critical when many values are missing.
- Principled Bayesian treatment automatically respects the observed-data likelihood and prior at every imputed draw.
- Works with modern probabilistic programming languages (Stan, PyMC, Turing.jl) that provide automatic differentiation and NUTS.
- Readily handles hierarchical and latent-variable models where multiple imputation by chained equations (MICE) would require custom imputation models.
- Requires differentiable log-posteriors; non-differentiable likelihoods (some discrete latent structures) break the gradient computation unless marginalised out.
- Computationally expensive per iteration relative to simpler samplers, and cost scales with the number of missing values added to the state vector.
- MAR must hold or a separate missingness model must be specified; the method does not automatically handle MNAR.
- Step-size and trajectory-length tuning is needed unless an adaptive sampler (NUTS) is used.
Frequently asked
How is this different from multiple imputation by chained equations (MICE)?
MICE imputes missing values iteratively using a sequence of conditional models and then analyzes each imputed dataset separately, combining results with Rubin's rules. HMC with missing data jointly samples parameters and missing values in a single principled model, avoiding the need for combining rules and fully propagating uncertainty. It is more coherent but requires a fully specified generative model and differentiable likelihoods.
Does this require me to model the missingness mechanism?
If missing data are MAR or MCAR, the missingness indicator can be ignored (ignorable missingness) and you only need to model the complete-data distribution. If data are MNAR, you must also specify a model for why values are missing, or results will be biased. HMC handles the sampling but does not diagnose the mechanism for you.
How do I check convergence when missing values are part of the state?
Include the imputed missing-value chains in standard diagnostics: compute R-hat and effective sample size (ESS) for a representative subset of imputed values alongside model parameters. R-hat above 1.01 or ESS below 400 for any component suggests the chain has not mixed, even if the parameter chains look fine.
Can I use NUTS (the No-U-Turn Sampler) here?
Yes. NUTS is the adaptive variant of HMC that automatically selects trajectory length and is the default sampler in Stan and PyMC. It works directly on the augmented state (parameters plus missing values) and typically requires no manual step-size tuning, making it the recommended implementation for practical use.
What proportion of missingness is too high for this method?
There is no hard threshold, but beyond roughly 40–50% missing at the item level the posterior becomes increasingly driven by the prior rather than the data, and imputation uncertainty dominates estimation. Sensitivity analyses with different priors on missing-data distributions are essential at high missingness rates.
Sources
- Neal, R. M. (2011). MCMC using Hamiltonian dynamics. In S. Brooks, A. Gelman, G. Jones & X.-L. Meng (Eds.), Handbook of Markov Chain Monte Carlo (pp. 113-162). CRC Press. ISBN: 978-1420079418
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A. & Rubin, D. B. (2013). Bayesian Data Analysis (3rd ed.). CRC Press. Chapter 18: Missing-data imputation. ISBN: 978-1439840955
How to cite this page
ScholarGate. (2026, June 3). Hamiltonian Monte Carlo with Missing Data Imputation. ScholarGate. https://scholargate.app/en/bayesian/hamiltonian-monte-carlo-with-missing-data
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Inference with Missing DataBayesian↔ compare
- Gibbs Sampling with Missing DataBayesian↔ compare
- Hamiltonian Monte CarloBayesian↔ compare
- MCMC with missing dataBayesian↔ compare
- Multiple ImputationStatistics↔ compare
- Variational Inference with Missing DataBayesian↔ compare