Variational Inference with Missing Data
Variational Bayesian Inference with Missing Data · Also known as: VI with missing data, variational EM with missing data, VB missing data, mean-field VI for incomplete data
Variational inference with missing data is a scalable Bayesian approach that simultaneously approximates the posterior over latent variables and model parameters while imputing missing observations. Instead of integrating over all possible values of the missing entries exactly, it posits a tractable approximate distribution and optimises it to be as close as possible to the true joint posterior, yielding fast, principled inference even in high-dimensional incomplete datasets.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use variational inference with missing data when the dataset is large enough to make full MCMC prohibitively slow, when data have a structured missing pattern (MAR or MCAR), and when you need both posterior inference over model parameters and principled imputation of missing values within a single coherent framework. It is especially valuable for latent-variable models (mixture models, factor analysis, deep generative models) applied to incomplete data. Avoid it when missingness is not at random (MNAR) and the missingness mechanism must be modelled explicitly without assuming independence, or when exact posterior accuracy is critical and computation budget allows MCMC — variational approximations can underestimate posterior variance.
Strengths & limitations
- Scales to large datasets and high-dimensional missing-data problems where MCMC is too slow.
- Jointly infers model parameters and imputes missing values within one principled objective.
- Provides a measurable optimisation criterion (ELBO) that tracks inference quality during training.
- Compatible with automatic differentiation and modern probabilistic programming systems (Pyro, NumPyro, Stan mean-field).
- Handles structured missingness patterns (MAR, MCAR) without ad-hoc imputation preprocessing.
- Mean-field factorisation underestimates posterior correlations and variance, potentially overconfident posteriors.
- Convergence to a local ELBO maximum is not guaranteed to be the global best approximation.
- Assumes missingness at random (MAR); MNAR settings require additional model components.
- Approximate imputed distributions may not reflect the true predictive uncertainty as faithfully as MCMC.
Frequently asked
How does this differ from multiple imputation followed by analysis?
Multiple imputation (e.g., MICE) imputes missing values first, then analyses each completed dataset separately, combining results by Rubin's rules. Variational inference with missing data integrates imputation and inference into one joint optimisation, so parameter uncertainty and missing-value uncertainty propagate to each other simultaneously rather than in two separate stages.
Does variational inference assume a specific missingness mechanism?
Standard formulations assume missing at random (MAR): the probability of a value being missing depends only on observed quantities, not on the missing value itself. If data are missing not at random (MNAR), an explicit model of the missingness process must be included, which complicates the ELBO and requires additional assumptions.
Can I use this with neural network models like VAEs?
Yes. Variational autoencoders (VAEs) are a direct application: the encoder approximates the posterior over a latent code and the decoder reconstructs observations. Extending VAEs to missing data — by marginalising the decoder over missing entries during training — is an active research area and works well in practice with partial-observation objectives.
How do I know if the variational approximation is accurate enough?
Check the ELBO trajectory for stable convergence, run posterior predictive checks on held-out observed entries, and if feasible compare moments of key posteriors against a short MCMC run. A large gap between ELBO and log-likelihood signals a poor approximation that may require a richer variational family.
What software supports this method?
Pyro and NumPyro (Python) support structured and mean-field VI with automatic differentiation. Stan offers mean-field ADVI. TensorFlow Probability and JAX-based libraries also provide flexible VI with missing-data support. For simpler conjugate models, custom CAVI implementations in Python or R are straightforward.
Sources
- Ghahramani, Z. & Jordan, M. I. (1994). Supervised learning from incomplete data via an EM approach. In Cowan, J. D., Tesauro, G. & Alspector, J. (Eds.), Advances in Neural Information Processing Systems 6 (pp. 120–127). Morgan Kaufmann. link ↗
- Wainwright, M. J. & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1–2), 1–305. DOI: 10.1561/2200000001 ↗
How to cite this page
ScholarGate. (2026, June 3). Variational Bayesian Inference with Missing Data. ScholarGate. https://scholargate.app/en/bayesian/variational-inference-with-missing-data
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Inference with Missing DataBayesian↔ compare
- Gibbs Sampling with Missing DataBayesian↔ compare
- MCMC with missing dataBayesian↔ compare
- Variational InferenceBayesian↔ compare