Bayesian Robust Regression
Also known as: Bayesian heavy-tailed regression, Bayesian Student-t regression, robust Bayesian linear model, BRR
Bayesian Robust Regression replaces the Gaussian error assumption of ordinary linear regression with a heavy-tailed distribution — most commonly the Student-t — and estimates all parameters in a Bayesian framework. The heavier tails give outliers less influence on the fitted line, yielding stable coefficient estimates and honest uncertainty intervals even when the data contain unusual observations.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Choose Bayesian Robust Regression when your outcome is continuous, you suspect outliers or heavy-tailed errors that would distort OLS, and you want full posterior uncertainty about the coefficients rather than just point estimates. It is especially valuable in small samples where outlier sensitivity is acute, in domains where extreme values are scientifically plausible (finance, environmental science, clinical measurements), or when you want to incorporate prior knowledge about coefficients. Do not use it when the Gaussian assumption is clearly satisfied and computational cost matters; OLS is simpler and equally valid then. Also avoid it if interpretability of the degrees-of-freedom parameter is critical without prior knowledge about tail behaviour.
Strengths & limitations
- Automatically down-weights outliers through the heavy-tailed Student-t likelihood, protecting coefficient estimates without ad hoc data cleaning.
- Provides full posterior distributions for all parameters, giving honest credible intervals rather than asymptotic confidence intervals.
- Degrees-of-freedom parameter nu is estimated from the data, letting the model learn how heavy the tails are rather than assuming a fixed value.
- Incorporates prior information naturally, which stabilises estimation in small samples or high-dimensional settings.
- Via the scale-mixture representation, inference is tractable with standard MCMC samplers and modern probabilistic programming languages.
- Computationally more expensive than OLS: MCMC sampling can be slow for large datasets or many predictors.
- Results depend on prior choices for nu and sigma; poorly chosen priors can dominate in small samples.
- Interpretation requires familiarity with posterior distributions and credible intervals, which may be unfamiliar to audiences trained only in frequentist methods.
- Does not select variables; a separate regularisation strategy or spike-and-slab prior is needed for sparse settings.
- Convergence of MCMC chains must be diagnosed (R-hat, trace plots), adding workflow overhead compared to OLS.
Frequently asked
How does Bayesian Robust Regression differ from frequentist M-estimation?
Both use heavy-tailed error models to down-weight outliers, but Bayesian robust regression provides full posterior distributions for coefficients and the degrees-of-freedom parameter, enabling credible intervals and prior incorporation. M-estimators give point estimates with asymptotic standard errors and do not propagate uncertainty about the error distribution itself.
What value should I use for the degrees-of-freedom prior?
A common choice is an exponential or shifted-exponential prior on nu with mean around 5–30, favouring mildly heavy tails while allowing the data to identify the true value. Fixing nu is discouraged unless domain knowledge justifies a specific tail weight.
Can I use this model for large datasets?
Yes, but full MCMC can be slow. Variational Bayes or mini-batch HMC (as in Stan's ADVI) can speed up inference. For very large n with limited outlier concern, OLS with heteroscedasticity-robust standard errors may be more practical.
Does down-weighting outliers mean the model ignores them?
No. Down-weighting reduces an outlier's pull on the regression line, but the observation still enters the likelihood and informs the posterior for all parameters, including nu. The model learns from outliers — it just does not let them dominate the fit.
How do I check whether the robust model fits better than OLS?
Compare models using leave-one-out cross-validation (LOO-CV) via the loo R package or arviz in Python, or use the widely applicable information criterion (WAIC). A lower expected log predictive density favours the better-fitting model.
Sources
- Geweke, J. (1993). Bayesian treatment of the independent Student-t linear model. Journal of Applied Econometrics, 8(S1), S19–S40. DOI: 10.1002/jae.3950080504 ↗
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian Data Analysis (3rd ed.). CRC Press. ISBN: 978-1439840955
How to cite this page
ScholarGate. (2026, June 3). Bayesian Robust Regression. ScholarGate. https://scholargate.app/en/statistics/bayesian-robust-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Generalized Linear ModelStatistics↔ compare
- Bayesian Multiple linear regressionStatistics↔ compare
- Bayesian Quantile RegressionStatistics↔ compare
- OLS RegressionEconometrics↔ compare
- Quantile RegressionEconometrics↔ compare
- Robust RegressionStatistics↔ compare