Bayesian Model Averaging
Also known as: BMA, Bayesian model combination, Bayesian Model Ortalaması (BMA)
Bayesian Model Averaging (BMA), formalised as a tutorial by Hoeting, Madigan, Raftery and Volinsky in 1999, addresses model uncertainty by averaging over all plausible model specifications rather than selecting a single best model. Each candidate model receives a posterior probability that reflects how well it fits the data given a prior, and predictions or coefficient estimates are formed as weighted averages across the entire model space. This approach reduces the bias and overconfidence that arise when a single selected model is treated as the true one.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+9 more
When to use it
BMA is appropriate when the set of relevant predictors is genuinely uncertain and selecting a single model would ignore that uncertainty. It performs best with n ≥ 50 and a moderate number of candidate predictors. It suits prediction problems where model-selection bias would otherwise inflate apparent precision, and exploratory analyses that need honest measures of variable importance. It is not the right choice when n is very small (posterior model probabilities are then dominated by the prior), or when a single transparent model is required for policy reporting.
Strengths & limitations
- Averages over model uncertainty rather than ignoring it, producing better-calibrated predictions.
- Posterior inclusion probabilities provide principled, probability-scaled measures of variable importance.
- Reduces the bias that arises from committing to a single selected model.
- With more than roughly 25 predictors, exact enumeration is infeasible and MCMC sampling of the model space is required, adding computational cost.
- Results are sensitive to the choice of prior on model parameters (e.g. unit information prior vs. diluted prior); this choice must be justified.
- When n < 50 the posterior model probabilities are unstable and unrepeatable.
Frequently asked
What is a posterior inclusion probability and how do I interpret it?
The posterior inclusion probability (PIP) for a predictor is the sum of posterior model probabilities across all models that include that predictor. A PIP close to 1 means the data strongly favour models with that predictor included; a PIP near 0 means it contributes little across the model space. PIPs are not p-values: there is no fixed threshold like 0.05, and the appropriate cut-off depends on the research question and prior.
How many predictors can BMA handle?
With k predictors the model space has 2ᵏ candidates. Exact enumeration is feasible up to roughly k = 20–25. Beyond that, MCMC model-space sampling (such as the birth-death sampler in the BMS package) is needed to focus computation on high-probability models without visiting all 2ᵏ.
Does BMA require normally distributed outcomes?
Standard BMA as implemented in packages like BMS assumes a Normal linear model for the outcomes. Non-Normal outcomes (binary, count) require extensions such as Bayesian logistic model averaging, which are less widely supported. When normality is implausible, a regularised frequentist approach such as LASSO or Elastic Net is often a more transparent alternative.
How is BMA different from simply selecting the best model by AIC or BIC?
AIC and BIC identify a single best model and discard all others, which understates uncertainty when several models fit similarly well. BMA retains all models and weights them by posterior probability, so the final inference honestly reflects model uncertainty. When one model is overwhelmingly better than all others, BMA effectively collapses to that single model.
Sources
- Hoeting, J. A., Madigan, D., Raftery, A. E. & Volinsky, C. T. (1999). Bayesian Model Averaging: A Tutorial. Statistical Science, 14(4), 382–401. link ↗
- Zeugner, S. & Feldkircher, M. (2015). Bayesian Model Averaging Employing Fixed and Flexible Priors: The BMS Package for R. Journal of Statistical Software, 68(4), 1–37. DOI: 10.18637/jss.v068.i04 ↗
How to cite this page
ScholarGate. (2026, June 1). Bayesian Model Averaging. ScholarGate. https://scholargate.app/en/bayesian/bayesian-model-averaging
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Hierarchical ModelBayesian↔ compare
- Bayesian RegressionBayesian↔ compare
- Elastic NetMachine learning↔ compare
- Lasso RegressionMachine learning↔ compare
- MCMCBayesian↔ compare