Bayesian Simple Linear Regression
Also known as: Bayesian SLR, Bayesian univariate regression, probabilistic simple linear regression, Bayesian linear model
Bayesian Simple Linear Regression models the relationship between a continuous outcome and a single predictor by combining a Gaussian likelihood with prior distributions over the intercept, slope, and error variance. The result is a full posterior distribution over all parameters, providing probabilistic uncertainty quantification rather than a single point estimate.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Bayesian Simple Linear Regression when you have a single continuous predictor and a continuous outcome and want principled uncertainty quantification, especially with small samples where frequentist confidence intervals are unreliable, or when prior subject-matter knowledge should be formally incorporated. It is also valuable when you need a full predictive distribution rather than point predictions. Avoid it when the computation overhead of MCMC is prohibitive, when no meaningful prior information exists and the sample is large (OLS will give nearly identical estimates at far less cost), or when interpretability for non-statistical audiences is the primary concern and posterior distributions would confuse rather than inform.
Strengths & limitations
- Produces a full posterior distribution over parameters, giving honest and complete uncertainty quantification.
- Naturally incorporates prior domain knowledge through prior distributions.
- Performs well in small samples where OLS standard errors are unreliable.
- Posterior predictive distributions enable principled uncertainty propagation into forecasts.
- Easy to extend hierarchically or to more complex likelihood structures without changing the inferential framework.
- Posterior probability statements (e.g., P(beta > 0) = 0.95) are more directly interpretable than p-values.
- Posterior inference via MCMC is computationally more demanding than OLS, especially for large datasets.
- Prior specification requires care; poorly chosen priors can dominate the likelihood in small samples.
- Convergence diagnostics (R-hat, effective sample size) must be checked, adding complexity to the workflow.
- Results can be harder to communicate to audiences unfamiliar with Bayesian reasoning.
- Conjugate closed-form solutions only exist for specific prior families (Normal-Inverse-Gamma).
Frequently asked
How does Bayesian Simple Linear Regression differ from OLS?
OLS minimises the sum of squared residuals and returns point estimates with frequentist confidence intervals. Bayesian regression places priors on all parameters and returns a full posterior distribution. With large samples and weakly informative priors the two give nearly identical coefficient estimates, but the Bayesian approach additionally yields posterior probabilities and propagates uncertainty into predictions.
What priors should I choose for the slope and intercept?
Weakly informative priors such as Normal(0, 10) for the slope and intercept are a common default. If you have subject-matter knowledge — say, from a prior study estimating the slope at around 2 — you can center the prior there and tighten the standard deviation. Always check sensitivity by re-fitting with alternative priors.
Do I need MCMC for Bayesian Simple Linear Regression?
Not always. With a Normal-Inverse-Gamma conjugate prior, the posterior is available in closed form and can be computed analytically. For other prior choices, MCMC (e.g., Stan, brms) or variational inference is needed.
What is a posterior predictive check?
A posterior predictive check generates new datasets by drawing parameters from the posterior and then simulating data from the likelihood. Comparing these simulated datasets to the observed data reveals whether the model captures the main features of the data — it is the Bayesian analogue of residual diagnostics.
When is a credible interval narrower or wider than an OLS confidence interval?
With an informative prior that is consistent with the data, the credible interval can be narrower than the frequentist confidence interval because prior information is incorporated. With a weakly informative prior and a large sample, both intervals are nearly identical. With a strongly informative but incorrect prior, the credible interval can be misleadingly narrow — prior misspecification is a real risk.
Sources
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian Data Analysis (3rd ed.). CRC Press. ISBN: 978-1439840955
- McElreath, R. (2020). Statistical Rethinking: A Bayesian Course with Examples in R and Stan (2nd ed.). CRC Press. ISBN: 978-0367139919
How to cite this page
ScholarGate. (2026, June 3). Bayesian Simple Linear Regression. ScholarGate. https://scholargate.app/en/statistics/bayesian-simple-linear-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Generalized Linear ModelStatistics↔ compare
- Bayesian Multiple linear regressionStatistics↔ compare
- Bayesian Quantile RegressionStatistics↔ compare
- Bayesian Robust RegressionStatistics↔ compare
- OLS RegressionEconometrics↔ compare
- Simple Linear RegressionStatistics↔ compare