Robust Bayesian Model Averaging
Also known as: robust BMA, outlier-robust BMA, robust model averaging, heavy-tailed BMA
Robust Bayesian model averaging extends standard BMA by replacing sensitive conjugate priors with heavy-tailed or mixture priors (e.g., mixtures of g-priors), and optionally robust likelihoods, so that posterior model probabilities and averaged estimates remain stable when data contain outliers, influential observations, or when the prior on model parameters would otherwise dominate the results.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use robust BMA when you want to average over model uncertainty but your data may contain outliers or influential points that could distort standard BMA results, or when the choice of the prior scale parameter (g) is unclear and you want the data to inform it. It is also appropriate in variable-selection problems with many candidate predictors where standard g-priors are known to produce Bartlett-paradox behavior. Do not use it as a first-choice replacement for standard BMA when data are clean and well-behaved — the extra complexity adds computational cost without benefit in those settings.
Strengths & limitations
- Propagates both parameter uncertainty and model uncertainty into every estimate and prediction.
- Heavy-tailed or mixture priors mitigate sensitivity to outliers and to arbitrary prior-scale choices.
- Avoids the Bartlett paradox that plagues fixed-g priors by letting g be data-adaptive.
- Applicable across regression, variable selection, and forecasting settings.
- Posterior model probabilities provide an interpretable ranking of competing models.
- Summing over 2^p models is computationally infeasible for large p; Markov chain Monte Carlo model composition (MC3) or stochastic search is required.
- Posterior model probabilities are sensitive to the model-space prior, especially when p is large relative to n.
- Interpretation of the averaged estimate can be difficult when top models differ substantially in their included predictors.
- Robust priors add a hyperparameter layer that itself requires justification.
Frequently asked
What makes this 'robust' compared to standard BMA?
Standard BMA with a fixed g-prior assigns a specific scale to the coefficient prior; a poor choice or a few outliers can dominate the marginal likelihoods, shifting posterior model weights unreliably. Robust BMA uses priors with heavier tails or data-adaptive scales (e.g., mixtures of g-priors) so the marginal likelihoods are less sensitive to these choices.
What is the Bartlett paradox and why does it matter here?
When the prior variance on coefficients goes to infinity (g → ∞ in a g-prior), the marginal likelihood of every non-null model collapses to that of the null model, making it impossible to identify any variable as important. Robust approaches fix this by using a proper mixture prior over g, so the marginal likelihood remains informative.
How do I handle the 2^p model-space sum in practice?
For p beyond roughly 20–25, exact enumeration is infeasible. Markov chain Monte Carlo model composition (MC3) uses a random walk over the model space, visiting high-probability models more frequently. The visited models approximate the full posterior model distribution.
Should I report BMA estimates or the single best model?
BMA estimates are preferable for prediction and for quantities that aggregate information across models. If the goal is a single interpretable model and BMA is dominated by one model anyway, reporting that model is reasonable — but the BMA posterior model probabilities should still be checked and reported.
Does robust BMA work with non-linear or non-Gaussian models?
Yes, the principle extends to generalized linear models and beyond, but computing marginal likelihoods requires Laplace approximations or thermodynamic integration rather than closed-form expressions, substantially increasing computational cost.
Sources
- Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999). Bayesian model averaging: A tutorial. Statistical Science, 14(4), 382–401. link ↗
- Ley, E., & Steel, M. F. J. (2012). Mixtures of g-priors for Bayesian model averaging with economic applications. Journal of Econometrics, 171(2), 251–266. link ↗
How to cite this page
ScholarGate. (2026, June 3). Robust Bayesian Model Averaging. ScholarGate. https://scholargate.app/en/bayesian/robust-bayesian-model-averaging
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Model AveragingBayesian↔ compare
- Bayesian RegressionBayesian↔ compare
- Hierarchical Bayesian InferenceBayesian↔ compare
- MCMCBayesian↔ compare
- Robust Bayesian InferenceBayesian↔ compare
- Variational InferenceBayesian↔ compare