Robust Simple Linear Regression
Also known as: robust SLR, M-estimator simple regression, outlier-resistant simple regression, robust bivariate regression
Robust simple linear regression fits a straight line through bivariate data using loss functions or weighting schemes that down-weight outliers, producing slope and intercept estimates that are far less sensitive to extreme observations than ordinary least squares while remaining easy to interpret.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Choose robust simple linear regression when you have a continuous outcome and one continuous predictor and suspect that the dataset contains outliers or leverage points that would distort an OLS fit. It is particularly valuable in small-to-moderate samples where a single extreme observation can shift the OLS line substantially. Do not use it as a routine replacement for OLS when data are clean and approximately normally distributed — the efficiency loss under ideal conditions is real. It is also not designed to handle heteroscedasticity (use weighted least squares) or non-linear relationships (use splines or GAMs).
Strengths & limitations
- Highly resistant to the distorting effect of outliers and leverage points on slope estimates.
- Retains the interpretability of OLS: beta1 is still the estimated change in y per unit increase in x.
- Asymptotically as efficient as OLS when errors are truly normal; relatively little is lost in clean data.
- Automatically identifies influential observations through the final IRLS weights, aiding diagnostics.
- MM-estimators achieve both high breakdown point (up to 50%) and high Gaussian efficiency simultaneously.
- More computationally intensive than OLS; requires iterative fitting and may not converge with very small samples.
- Standard errors require sandwich or bootstrap methods; classical OLS F and t tables do not apply directly.
- Does not correct for heteroscedasticity — non-constant error variance must be handled separately.
- Results depend on the choice of tuning constant and influence function (Huber, Tukey bisquare, etc.); software defaults may not suit every application.
Frequently asked
How is robust simple linear regression different from OLS?
OLS minimises the sum of squared residuals, which makes it very sensitive to outliers. Robust regression minimises a bounded or slower-growing loss function, so extreme observations receive less weight and have far less influence on the estimated slope and intercept.
What is the tuning constant and how do I choose it?
The tuning constant (e.g., kappa = 1.345 for Huber loss) controls the threshold at which residuals transition from the quadratic to the linear penalty regime. The default 1.345 gives 95% asymptotic efficiency relative to OLS under normal errors. Smaller values are more robust but less efficient; larger values approach OLS.
Can robust regression handle heteroscedasticity?
No — robust regression addresses outliers in the response, not non-constant error variance. For heteroscedasticity use weighted least squares or heteroscedasticity-consistent (sandwich) standard errors applied to OLS.
How do I interpret the final IRLS weights?
A weight near 1 means the observation is consistent with the bulk of the data; a weight near 0 means it is an outlier that the estimator has effectively excluded. Printing these weights is an excellent diagnostic for identifying unusual cases.
What is the breakdown point and why does it matter?
The breakdown point is the fraction of contaminated observations an estimator can tolerate before producing arbitrarily wrong estimates. OLS has a 0% breakdown point. Huber M-estimators reach around 10-20%, while MM-estimators and least trimmed squares can reach 50%, meaning up to half the data can be contaminated before the estimate fails.
Sources
- Rousseeuw, P. J., & Leroy, A. M. (1987). Robust Regression and Outlier Detection. John Wiley & Sons. ISBN: 978-0471852339
- Huber, P. J. (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1), 73-101. DOI: 10.1214/aoms/1177703732 ↗
How to cite this page
ScholarGate. (2026, June 3). Robust Simple Linear Regression. ScholarGate. https://scholargate.app/en/statistics/robust-simple-linear-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- OLS RegressionEconometrics↔ compare
- Quantile RegressionEconometrics↔ compare
- Robust Multiple linear regressionStatistics↔ compare
- Robust RegressionStatistics↔ compare
- Theil-Sen EstimatorStatistics↔ compare
- Weighted Least SquaresStatistics↔ compare