Bayesian Linear Regression
Also known as: bayesian linear model, probabilistic linear regression, Bayesçi Doğrusal Regresyon
Bayesian linear regression is a probabilistic extension of the ordinary linear model, introduced through Bayes' rule and formalised in its modern computational workflow by Gelman et al. (2013). Rather than returning a single point estimate for each coefficient, it combines a user-specified prior distribution with the likelihood of the observed data to produce a full posterior distribution over all parameters, from which credible intervals and posterior predictive distributions are derived.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Bayesian linear regression is most valuable when the sample is small or sparse, because a sensible prior stabilises coefficient estimates where data alone would be insufficient. It is the natural choice when genuine prior knowledge exists and should be incorporated formally, or when the full uncertainty of every coefficient and prediction is needed as a distribution rather than a point estimate with a standard error. The method requires that the prior distribution be specified and justified, that MCMC convergence be verified via R-hat and effective sample size, and that results be reported as credible intervals. When the sample is large and no prior information is available, ordinary frequentist regression typically gives equivalent answers with less computation.
Strengths & limitations
- Returns a full posterior distribution for each coefficient, providing credible intervals that carry a direct probability interpretation.
- Incorporates prior knowledge formally, making the method robust in small or sparse samples.
- Produces posterior predictive distributions for new observations, giving calibrated uncertainty for forecasts.
- Works without the strict normality requirement imposed by frequentist inference, as the Bayesian framework is not reliant on asymptotic theory in the same way.
- When the sample is very small (n close to 10), the posterior is dominated by the prior and the data contribute little information.
- Prior choice can materially influence results; if prior-sensitivity analysis shows the posterior shifts noticeably with reasonable prior changes, the conclusions are not robust.
- Computationally more demanding than ordinary least-squares regression, because the posterior must be sampled or approximated rather than solved analytically in the general case.
- Requires the analyst to specify and justify prior distributions, adding a layer of judgment that frequentist methods do not.
Frequently asked
What is the difference between a credible interval and a confidence interval?
A 95% credible interval is a direct probability statement: given the data and the prior, there is a 95% probability that the parameter lies within that range. A frequentist 95% confidence interval is a statement about long-run procedure coverage — if the same experiment were repeated many times, 95% of such intervals would contain the true value — not a probability statement about any single interval. Reading a credible interval as a confidence interval, or vice versa, is a common and consequential error.
How should I choose a prior distribution?
Use an informative prior when you have genuine prior knowledge to encode — for example, from a previous meta-analysis or domain expertise. Use a weakly informative prior when you mostly want to let the data speak while preventing the sampler from exploring implausible regions. Whichever prior you choose must be explicitly justified in your report, and you should run a prior-sensitivity analysis: if the posterior shifts substantially with reasonable changes to the prior, the conclusion is not robust.
What are MCMC, NUTS, and R-hat, and why do they matter?
Markov chain Monte Carlo (MCMC) is a family of algorithms for drawing samples from the posterior when it cannot be computed analytically. NUTS (No-U-Turn Sampler) and Hamiltonian Monte Carlo are efficient MCMC algorithms widely used in Bayesian regression software. R-hat (also written R̂) compares the variance within chains to the variance between chains; values near 1.0 — typically below 1.01 — indicate that the chains have converged to the same distribution. An adequate effective sample size (ESS) is also required. Without convergence checks, the posterior summaries cannot be trusted.
When does Bayesian linear regression agree with ordinary least squares?
With a large sample and flat (non-informative) priors, the posterior mean and 95% credible interval of each coefficient typically coincide closely with the OLS point estimate and 95% confidence interval. The two methods diverge when the sample is small, when an informative prior is used, or when the analyst needs the full posterior distribution rather than a single-number summary.
Sources
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A. & Rubin, D. B. (2013). Bayesian Data Analysis (3rd ed.). CRC Press. ISBN: 978-1439840955
How to cite this page
ScholarGate. (2026, June 1). Bayesian Linear Regression. ScholarGate. https://scholargate.app/en/bayesian/bayesian-linear-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian ANOVABayesian↔ compare
- Bayesian RegressionBayesian↔ compare
- MCMCBayesian↔ compare
- OLS RegressionEconometrics↔ compare