Bayesian Information Criterion (BIC)
Also known as: BIC, Schwarz criterion, Schwarz information criterion
The Bayesian Information Criterion is an information-theoretic model selection criterion that approximates Bayesian model comparison. Introduced by Gideon Schwarz in 1978, BIC penalizes model complexity more heavily than AIC by using a sample-size-dependent penalty, making it particularly suitable for identifying the true underlying model structure.
Key highlights
- Consistent: asymptotically selects the true model under correct specification
- Stronger overfitting prevention: penalizes complexity more than AIC
- Grounded in Bayesian theory: approximates Bayesian model averaging
- Useful for variable selection: tends to exclude irrelevant predictors
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use BIC when comparing models and you suspect one model is the true data-generating process. BIC is particularly valuable for variable selection and model comparison in regression, time-series, and classification. BIC strongly penalizes complexity; use it when parsimony is valued (e.g., interpretability in economics or epidemiology). With large samples, BIC strongly prefers simpler models over AIC. For prediction-focused tasks, prefer AIC.
Strengths & limitations
- Consistent: asymptotically selects the true model under correct specification
- Stronger overfitting prevention: penalizes complexity more than AIC
- Grounded in Bayesian theory: approximates Bayesian model averaging
- Useful for variable selection: tends to exclude irrelevant predictors
- Assumes the true model is in the candidate set; inconsistent otherwise
- Very large penalty for models with many parameters; may be too aggressive
- Like AIC, only relative values are interpretable; absolute values meaningless
- Requires full likelihood specification; not applicable to all models
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How do I interpret BIC differences?
As a rough guide: delta BIC < 2 suggests weak evidence against higher BIC model, 2-6 is positive evidence, 6-10 is strong evidence, > 10 is very strong evidence. These are guidelines; context and prior knowledge matter.
When should I use BIC instead of AIC?
Use BIC when you want a simpler, more interpretable model (explanatory goals). Use AIC when you prioritize prediction accuracy. With large samples, BIC becomes much more conservative than AIC; BIC may reject complex models that AIC accepts.
How does sample size affect BIC?
The penalty k log(n) grows with n. For large samples, BIC heavily penalizes extra parameters; even small improvements in fit do not justify added complexity. This property makes BIC consistent (selects true model asymptotically) but can be too strict on small samples.
Can I use BIC for non-nested model comparison?
Yes, BIC can compare any two models fit to the same data. However, interpretation is more complex than AIC because the penalty structure differs. Always report BIC differences and use domain knowledge.
Is BIC approximating Bayesian model comparison?
BIC approximates the log Bayes factor under specific prior assumptions (uniform priors on parameters). More sophisticated Bayesian methods (posterior model probability) may be preferable for formal Bayesian analysis.
Sources
- 1.Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461-464.
- 2.Burnham, K. P., & Anderson, D. R. (2002). Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach (2nd ed.). New York: Springer.
- 3.Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90(430), 773-795.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Bayesian Information Criterion. ScholarGate. https://scholargate.app/model-evaluation/bayesian-information-criterion