Bayesian OLS (Bayesian Ordinary Least Squares Regression)
Also known as: Bayesian linear regression, Bayesian normal regression, BLR, Bayesian least squares
Bayesian OLS combines the classical linear regression likelihood with prior distributions over the coefficients and error variance. Rather than reporting point estimates, it produces full posterior distributions that quantify both estimated effects and their uncertainty. The approach is especially valuable when prior knowledge is available or when samples are small.
Key highlights
- Produces full posterior distributions over coefficients, giving richer uncertainty quantification than point estimates and p-values alone.
- Formally incorporates prior information from theory or earlier studies, improving precision when samples are small.
- Under a diffuse prior, results are numerically close to classical OLS, providing a natural bridge between frequentist and Bayesian inference.
- Conjugate Normal-Inverse-chi-squared priors yield closed-form posteriors, making computation fast even without MCMC.
- Credible intervals have a direct probability interpretation: there is 95% posterior probability that β lies in the reported interval.
- Enables coherent Bayesian model comparison via Bayes factors or marginal likelihoods.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use Bayesian OLS when you have a continuous outcome and a linear model but also credible prior information (e.g., from earlier studies or theoretical constraints) that you want to incorporate formally. It is especially advantageous with small samples where OLS estimates are unstable, when you need coherent uncertainty quantification (credible intervals rather than frequentist confidence intervals), or when regularisation through an informative prior is preferable to ad-hoc penalty terms. Avoid it when the prior is hard to defend and reviewers may challenge subjectivity, or when the model is highly nonlinear and conjugacy breaks down — in that case full MCMC-based Bayesian regression is the proper extension. Do not use it simply to dress up classical OLS in Bayesian language without genuine prior elicitation.
Strengths & limitations
- Produces full posterior distributions over coefficients, giving richer uncertainty quantification than point estimates and p-values alone.
- Formally incorporates prior information from theory or earlier studies, improving precision when samples are small.
- Under a diffuse prior, results are numerically close to classical OLS, providing a natural bridge between frequentist and Bayesian inference.
- Conjugate Normal-Inverse-chi-squared priors yield closed-form posteriors, making computation fast even without MCMC.
- Credible intervals have a direct probability interpretation: there is 95% posterior probability that β lies in the reported interval.
- Enables coherent Bayesian model comparison via Bayes factors or marginal likelihoods.
- Prior specification requires justification; poorly chosen priors can distort results, especially with small samples.
- Conjugate closed-form posteriors assume Gaussian errors; non-Gaussian likelihoods or hierarchical structures require MCMC sampling, which is computationally heavier.
- Results depend on the chosen prior, introducing subjectivity that may invite criticism in applied or regulatory contexts.
- Communicating posterior distributions and credible intervals to audiences trained in frequentist methods can be challenging.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How does Bayesian OLS differ from classical OLS?
Classical OLS produces a single point estimate for each coefficient and assesses uncertainty via sampling distributions under repeated experiments. Bayesian OLS treats coefficients as random variables, combines the likelihood with a prior, and reports a posterior distribution. Under a diffuse prior the posterior mean equals the OLS estimate, but uncertainty is quantified as a credible interval with a direct probability interpretation rather than a frequentist confidence interval.
What prior should I use if I have no strong prior beliefs?
A diffuse or non-informative prior — for example a flat prior on β and Jeffreys prior on σ² — lets the data dominate and recovers results very close to classical OLS. If you have even weak theoretical knowledge about coefficient signs or magnitudes, encoding that as a mildly informative Normal prior typically improves precision without imposing strong constraints.
Is Bayesian OLS the same as ridge regression?
They are related but distinct. Ridge regression is equivalent to Bayesian OLS with a zero-mean Normal prior on β whose variance is proportional to σ²/λ, where λ is the ridge penalty. Ridge is typically justified as a frequentist regularisation device and reports point estimates only, while Bayesian OLS explicitly reports the full posterior distribution and requires principled prior elicitation.
Do I need MCMC to run Bayesian OLS?
Not under the standard Normal-Inverse-Gamma conjugate prior, which yields a closed-form analytic posterior. MCMC is only required when you use non-conjugate priors, include nonlinear terms, add hierarchical structure, or impose sign restrictions that make the posterior intractable analytically.
How do I compare competing Bayesian OLS models?
The standard tool is the Bayes factor — the ratio of marginal likelihoods of two models — which quantifies evidence in favour of one specification over another on a continuous scale. For model selection across many candidates, the Deviance Information Criterion (DIC) or Leave-One-Out cross-validation (LOO-CV) are practical alternatives that do not require computing marginal likelihoods directly.
Sources
- 1.Zellner, A. (1971). An Introduction to Bayesian Inference in Econometrics. Wiley.ISBN 978-0471169376
- 2.Koop, G. (2003). Bayesian Econometrics. Wiley-Interscience.ISBN 978-0470845677
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Bayesian OLS. ScholarGate. https://scholargate.app/econometrics/bayesian-ols