Akaike Information Criterion (AIC)
Also known as: AIC
The Akaike Information Criterion is an information-theoretic measure for model selection that balances goodness of fit against model complexity. Introduced by Hirotugu Akaike in 1974, AIC estimates the relative quality of models for a given dataset, penalizing additional parameters to prevent overfitting.
Key highlights
- Information-theoretic foundation: estimates relative quality of out-of-sample prediction
- Works with non-nested models: directly comparable across different model structures
- Prevents overfitting by penalizing complexity
- Consistent with maximum likelihood estimation framework
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use AIC when comparing multiple candidate models fit to the same dataset. It is particularly valuable for model selection in regression, time-series, and mixed-effect contexts. AIC assumes the true model is in your candidate set and is most useful for comparing non-nested models. For very large sample sizes, the penalty 2k may be too lenient; consider BIC instead. AIC is prediction-focused, not fit-focused.
Strengths & limitations
- Information-theoretic foundation: estimates relative quality of out-of-sample prediction
- Works with non-nested models: directly comparable across different model structures
- Prevents overfitting by penalizing complexity
- Consistent with maximum likelihood estimation framework
- Only relative values matter; absolute AIC is uninterpretable
- Assumes observations are independent; violates with time-series autocorrelation
- Requires full likelihood specification; not applicable to all model types
- Bias toward complex models with large samples; penalty 2k stays constant
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How many points AIC difference is meaningful?
As a rough guide: delta AIC < 2 suggests negligible difference, 4-7 suggests less support for the higher AIC model, > 10 suggests very little support. However, context matters; always use domain knowledge in addition to AIC rankings.
Should I use AIC or BIC?
AIC is prediction-focused; BIC is explanation-focused. AIC tends to select more complex models (as sample size increases, penalty stays 2k), while BIC penalty grows with n log(n). Use AIC for prediction, BIC for understanding true model structure.
Can I use AIC with time-series data?
With caution. AIC assumes independence; time-series data violate this. Some time-series models (ARIMA, GARCH) still provide proper likelihoods, making AIC valid. However, auto-correlated residuals can bias AIC comparisons.
Why does AIC increase with sample size for fixed k?
As n grows, the likelihood magnitude changes (more observations, larger product). This is expected; AIC values across datasets are not comparable. Only compare AIC within the same dataset.
How does AIC relate to cross-validation?
Both estimate out-of-sample prediction error. AIC is faster (no need to refit), but cross-validation is more direct and requires fewer assumptions. For critical decisions, use both methods.
Sources
- 1.Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716-723.
- 2.Burnham, K. P., & Anderson, D. R. (2002). Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach (2nd ed.). New York: Springer.
- 3.Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. Annals of Mathematical Statistics, 22(1), 79-86.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Akaike Information Criterion. ScholarGate. https://scholargate.app/model-evaluation/akaike-information-criterion