Multilevel MCMC
Multilevel Markov Chain Monte Carlo · Also known as: hierarchical MCMC, multilevel Bayesian sampling, MLMCMC, hierarchical Markov chain Monte Carlo
Multilevel MCMC applies Markov chain Monte Carlo sampling to hierarchical (multilevel) Bayesian models. It draws samples from the joint posterior of both group-level and population-level parameters simultaneously, propagating uncertainty across levels and enabling inference in clustered or nested data structures where observations within groups share common distributional characteristics.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+3 more
When to use it
Use Multilevel MCMC when data are naturally clustered or nested (students in schools, patients in hospitals, measurements within subjects) and you need full Bayesian uncertainty for both group-level and population-level parameters. It is especially valuable when group sample sizes vary widely — it borrows strength from larger groups to stabilise estimates for smaller ones. Use it when prior information about variance components exists or when the research goal is to partition variability across levels. Do not use it when observations are independent with no grouping structure (ordinary Bayesian regression suffices), when run-time is severely constrained and a frequentist mixed-effects model gives an adequate approximation, or when group counts are very small (fewer than five groups) and hyperparameter estimation is unreliable.
Strengths & limitations
- Propagates uncertainty across all levels of the hierarchy simultaneously, yielding calibrated credible intervals for both group-specific and population-level parameters.
- Partial pooling stabilises estimates for small or data-sparse groups without discarding information from larger groups.
- Naturally accommodates imbalanced designs, missing data, and non-standard outcome distributions (e.g. Poisson, Bernoulli) within the same framework.
- Full posterior enables principled model comparison (via LOO-CV or WAIC) and posterior predictive checks.
- Compatible with modern probabilistic programming languages (Stan, PyMC, JAGS) that handle complex hierarchical structures automatically.
- Computationally expensive: large hierarchies with many groups and parameters require substantial sampling time, especially when gradients are unavailable.
- Posterior geometry in multilevel models often exhibits funnel-shaped dependencies that slow mixing; non-centred parameterisation helps but requires expertise.
- Hyperpriors on variance components (e.g. the between-group SD) are influential and must be chosen carefully, as weakly informative choices can still bias results.
- With very few groups (J < 5) the population distribution is poorly identified and the posterior of hyperparameters is dominated by the prior.
- Convergence assessment is more demanding than in flat models because both group-level and hyperparameter chains must be checked.
Frequently asked
How does Multilevel MCMC differ from running separate MCMC analyses for each group?
Separate analyses ignore the population structure, so each group is estimated using only its own data — small groups get very uncertain estimates. Multilevel MCMC estimates all groups jointly under a shared population distribution, which regularises small-group estimates by borrowing strength from the rest. The amount of pooling is determined by the data, not fixed in advance.
What is non-centred parameterisation and when do I need it?
In the centred form, group parameters theta_j are directly sampled from N(mu, sigma). When sigma is small the theta_j are tightly constrained and the sampler must navigate a narrow funnel, causing poor mixing. The non-centred form reparameterises as theta_j = mu + sigma * z_j and samples the standardised offsets z_j, which are independent of sigma. This geometry is far easier for NUTS and is recommended whenever hierarchical variance components are uncertain.
How many groups do I need for multilevel MCMC to work well?
With five or more groups the hyperparameters become reasonably identifiable; with twenty or more the population distribution is estimated reliably. Below five groups, the between-group variance is poorly identified and results are strongly prior-dependent. In such cases a fixed-effects model or strong informative hyperprior should be considered instead.
How do I assess whether the multilevel structure is necessary?
Compute the intraclass correlation coefficient (ICC) from the posterior. A near-zero ICC indicates little between-group variability and a flat model may suffice. LOO-CV or WAIC comparison between a flat Bayesian model and the hierarchical model provides a principled predictive criterion for model selection.
Can Multilevel MCMC handle non-normal outcomes?
Yes. The MCMC framework places no restriction on the likelihood: Bernoulli/logistic for binary outcomes, Poisson or negative-binomial for counts, Weibull for survival times, and so on are all straightforward to specify in Stan or PyMC. The hierarchical structure is independent of the distributional form of the observations.
Sources
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A. & Rubin, D. B. (2013). Bayesian Data Analysis (3rd ed.). CRC Press. ISBN: 978-1439840955
- Gelfand, A. E. & Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85(410), 398-409. DOI: 10.1080/01621459.1990.10476213 ↗
How to cite this page
ScholarGate. (2026, June 3). Multilevel Markov Chain Monte Carlo. ScholarGate. https://scholargate.app/en/bayesian/multilevel-mcmc
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian RegressionBayesian↔ compare
- Gibbs SamplingBayesian↔ compare
- Hamiltonian Monte CarloBayesian↔ compare
- Hierarchical Bayesian InferenceBayesian↔ compare
- Metropolis-Hastings AlgorithmBayesian↔ compare
- Variational InferenceBayesian↔ compare