Bayesian Regression
Bayesian Linear Regression · Also known as: bayesian linear regression, probabilistic regression, bayesian regresyon
Bayesian regression is a probabilistic version of linear regression that treats the model parameters as uncertain quantities. Instead of returning a single best-fit estimate, it combines prior knowledge with the observed data to produce a full posterior probability distribution for each parameter, from which credible intervals and predictions are read off.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+45 more
When to use it
Bayesian regression is a strong choice when the sample is small or sparse, because a sensible prior stabilises the estimates where the data alone would be too thin. It is also the natural option when genuine prior information exists and should be folded into the analysis, or when you need the full uncertainty of every parameter and prediction expressed as a distribution rather than a single point with a standard error. When the sample is large and no prior information is available, ordinary frequentist regression usually gives very similar answers more cheaply.
Strengths & limitations
- Reports full parameter uncertainty as a posterior distribution with credible intervals, not just a point estimate.
- Incorporates prior knowledge and remains usable in small or sparse-data settings.
- Predictions come with their own posterior distribution, giving honest uncertainty for forecasts.
- When n is small the posterior is dominated by the prior and the data adds little information.
- Results are unreliable if prior-sensitivity analysis shows the posterior shifts with the prior choice.
- Computationally heavier than frequentist fits, since the posterior must be sampled or approximated.
Frequently asked
What is the difference between a credible interval and a confidence interval?
A 95% credible interval is a direct probability statement: given the data and prior, there is a 95% probability the parameter lies in that range. A frequentist confidence interval is a statement about the long-run coverage of the procedure, not about a single interval. Reading a credible interval as a confidence interval (or vice versa) is a common mistake.
How do I choose a prior?
Use an informative prior when you have genuine prior knowledge to encode, and a weakly-informative prior when you mostly want to let the data speak while keeping estimates stable. Whatever you pick must be justified, and you should run a prior-sensitivity analysis — if the posterior shifts noticeably with reasonable prior changes, the result is not robust.
What are MCMC and R-hat?
MCMC (Markov chain Monte Carlo, e.g. NUTS / Hamiltonian Monte Carlo) draws samples from the posterior when it cannot be computed directly. R-hat is a convergence diagnostic that compares multiple chains; values below about 1.01, together with an adequate effective sample size, indicate the chains have mixed and the posterior summaries can be trusted.
How does Bayesian regression differ from OLS?
Ordinary least squares returns a single point estimate per coefficient with a standard error. Bayesian regression returns a full posterior distribution shaped by both the data and a prior, so it naturally handles small samples and prior information. With large samples and flat priors the two tend to agree closely.
Sources
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A. & Rubin, D. B. (2013). Bayesian Data Analysis (3rd ed.). CRC Press. ISBN: 978-1439840955
How to cite this page
ScholarGate. (2026, June 1). Bayesian Linear Regression. ScholarGate. https://scholargate.app/en/bayesian/bayesian-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- MCMCBayesian↔ compare
- OLS RegressionEconometrics↔ compare