Hierarchical Hamiltonian Monte Carlo
Hamiltonian Monte Carlo for Hierarchical Models · Also known as: Hierarchical HMC, HMC for hierarchical models, HMC with reparameterization, NUTS for hierarchical Bayesian models
Hierarchical Hamiltonian Monte Carlo (Hierarchical HMC) applies Hamiltonian Monte Carlo sampling to Bayesian hierarchical models, addressing the severe geometric challenges those models pose. By combining non-centered parameterizations with HMC's gradient-driven proposals, it achieves efficient posterior exploration of the multi-level funnel-shaped geometries that standard MCMC methods struggle with.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Hierarchical HMC when fitting Bayesian hierarchical or multilevel models where standard MCMC (Gibbs, random-walk Metropolis-Hastings) shows poor mixing, high autocorrelation, or divergent transitions — common signs of funnel geometry. It is the recommended sampler in Stan for most hierarchical specifications. Do not use it when the model has discrete latent variables (HMC requires continuous differentiable posteriors), when the posterior is low-dimensional and simple (overhead is unnecessary), or when gradient computation is prohibitively expensive per step.
Strengths & limitations
- Efficiently explores the funnel-shaped posteriors characteristic of hierarchical models, where other MCMC methods fail.
- Non-centered reparameterization combined with HMC often yields near-independent posterior samples with high effective sample size.
- Divergent transitions serve as a diagnostic: they signal geometric pathologies before results are trusted.
- Implemented in Stan and PyMC with automatic differentiation, making it accessible without manual gradient derivation.
- Scales well to hundreds of parameters compared to random-walk methods.
- Requires a continuous, differentiable log-posterior; incompatible with discrete latent variables.
- Gradient evaluation cost grows with model complexity and dataset size, making each iteration more expensive than simple MCMC.
- Tuning step size and path length (or NUTS adaptation) requires a warm-up phase that can be long for challenging geometries.
- Extremely hierarchically structured models with near-zero variance components may still exhibit problematic funnels even after non-centering.
Frequently asked
What is a divergent transition and why does it matter?
A divergent transition occurs when the leapfrog integrator loses energy conservation due to extreme curvature in the posterior — typically at the neck of a funnel. Divergences indicate that posterior regions are not being sampled correctly, so any reported estimates are biased. The fix is usually a non-centered reparameterization, tighter priors, or a smaller step size.
When should I use non-centered vs. centered parameterization?
Non-centered parameterization decouples the geometry and is preferred when group-level sample sizes are small and the data provide little information about individual groups. Centered parameterization can be better when groups are large and well-observed, because it concentrates the posterior more tightly. In practice, Stan's default or a partially-non-centered approach is recommended when in doubt.
How does Hierarchical HMC differ from standard HMC?
The underlying sampler is the same, but Hierarchical HMC specifically addresses the parameterization and tuning challenges that arise from hierarchical posterior geometry. Key additions are the non-centered reparameterization to flatten funnels, and monitoring divergent transitions as a geometry diagnostic unique to hierarchical settings.
Can I use Hierarchical HMC with discrete parameters?
No. HMC requires continuous, differentiable log-posteriors because it relies on gradients to simulate Hamiltonian dynamics. Discrete latent variables must be marginalized out analytically before applying HMC, or handled with a separate discrete sampler in a Gibbs-within-HMC scheme.
What software implements Hierarchical HMC?
Stan (via its NUTS sampler) is the primary tool and handles non-centered parameterization through model code. PyMC also implements NUTS with similar capabilities. Both provide R-hat, ESS, and divergence diagnostics automatically.
Sources
- Betancourt, M. & Girolami, M. (2015). Hamiltonian Monte Carlo for hierarchical models. In S. K. Upadhyay, U. Singh, D. K. Dey & A. Loganathan (Eds.), Current Trends in Bayesian Methodology with Applications (pp. 79-101). CRC Press. link ↗
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A. & Rubin, D. B. (2013). Bayesian Data Analysis (3rd ed.). CRC Press. ISBN: 978-1439840955
How to cite this page
ScholarGate. (2026, June 3). Hamiltonian Monte Carlo for Hierarchical Models. ScholarGate. https://scholargate.app/en/bayesian/hierarchical-hamiltonian-monte-carlo
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian RegressionBayesian↔ compare
- Hamiltonian Monte CarloBayesian↔ compare
- Hierarchical Bayesian InferenceBayesian↔ compare
- Hierarchical Markov Chain Monte CarloBayesian↔ compare
- MCMCBayesian↔ compare