Bayesian OLS (Bayesian Ordinary Least Squares Regression)
Bayesian Ordinary Least Squares Regression · Also known as: Bayesian linear regression, Bayesian normal regression, BLR, Bayesian least squares
Bayesian OLS combines the classical linear regression likelihood with prior distributions over the coefficients and error variance. Rather than reporting point estimates, it produces full posterior distributions that quantify both estimated effects and their uncertainty. The approach is especially valuable when prior knowledge is available or when samples are small.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Bayesian OLS when you have a continuous outcome and a linear model but also credible prior information (e.g., from earlier studies or theoretical constraints) that you want to incorporate formally. It is especially advantageous with small samples where OLS estimates are unstable, when you need coherent uncertainty quantification (credible intervals rather than frequentist confidence intervals), or when regularisation through an informative prior is preferable to ad-hoc penalty terms. Avoid it when the prior is hard to defend and reviewers may challenge subjectivity, or when the model is highly nonlinear and conjugacy breaks down — in that case full MCMC-based Bayesian regression is the proper extension. Do not use it simply to dress up classical OLS in Bayesian language without genuine prior elicitation.
Strengths & limitations
- Produces full posterior distributions over coefficients, giving richer uncertainty quantification than point estimates and p-values alone.
- Formally incorporates prior information from theory or earlier studies, improving precision when samples are small.
- Under a diffuse prior, results are numerically close to classical OLS, providing a natural bridge between frequentist and Bayesian inference.
- Conjugate Normal-Inverse-chi-squared priors yield closed-form posteriors, making computation fast even without MCMC.
- Credible intervals have a direct probability interpretation: there is 95% posterior probability that β lies in the reported interval.
- Enables coherent Bayesian model comparison via Bayes factors or marginal likelihoods.
- Prior specification requires justification; poorly chosen priors can distort results, especially with small samples.
- Conjugate closed-form posteriors assume Gaussian errors; non-Gaussian likelihoods or hierarchical structures require MCMC sampling, which is computationally heavier.
- Results depend on the chosen prior, introducing subjectivity that may invite criticism in applied or regulatory contexts.
- Communicating posterior distributions and credible intervals to audiences trained in frequentist methods can be challenging.
Frequently asked
How does Bayesian OLS differ from classical OLS?
Classical OLS produces a single point estimate for each coefficient and assesses uncertainty via sampling distributions under repeated experiments. Bayesian OLS treats coefficients as random variables, combines the likelihood with a prior, and reports a posterior distribution. Under a diffuse prior the posterior mean equals the OLS estimate, but uncertainty is quantified as a credible interval with a direct probability interpretation rather than a frequentist confidence interval.
What prior should I use if I have no strong prior beliefs?
A diffuse or non-informative prior — for example a flat prior on β and Jeffreys prior on σ² — lets the data dominate and recovers results very close to classical OLS. If you have even weak theoretical knowledge about coefficient signs or magnitudes, encoding that as a mildly informative Normal prior typically improves precision without imposing strong constraints.
Is Bayesian OLS the same as ridge regression?
They are related but distinct. Ridge regression is equivalent to Bayesian OLS with a zero-mean Normal prior on β whose variance is proportional to σ²/λ, where λ is the ridge penalty. Ridge is typically justified as a frequentist regularisation device and reports point estimates only, while Bayesian OLS explicitly reports the full posterior distribution and requires principled prior elicitation.
Do I need MCMC to run Bayesian OLS?
Not under the standard Normal-Inverse-Gamma conjugate prior, which yields a closed-form analytic posterior. MCMC is only required when you use non-conjugate priors, include nonlinear terms, add hierarchical structure, or impose sign restrictions that make the posterior intractable analytically.
How do I compare competing Bayesian OLS models?
The standard tool is the Bayes factor — the ratio of marginal likelihoods of two models — which quantifies evidence in favour of one specification over another on a continuous scale. For model selection across many candidates, the Deviance Information Criterion (DIC) or Leave-One-Out cross-validation (LOO-CV) are practical alternatives that do not require computing marginal likelihoods directly.
Sources
- Zellner, A. (1971). An Introduction to Bayesian Inference in Econometrics. Wiley. ISBN: 978-0471169376
- Koop, G. (2003). Bayesian Econometrics. Wiley-Interscience. ISBN: 978-0470845677
How to cite this page
ScholarGate. (2026, June 3). Bayesian Ordinary Least Squares Regression. ScholarGate. https://scholargate.app/en/econometrics/bayesian-ols
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Fixed Effects ModelEconometrics↔ compare
- Bayesian Random Effects ModelEconometrics↔ compare
- Bayesian VAR modelEconometrics↔ compare
- OLS RegressionEconometrics↔ compare
- Ridge RegressionMachine learning↔ compare