Robust Regression
Also known as: M-estimation regression, robust linear regression, outlier-resistant regression, MM-estimation
Robust regression estimates the linear relationship between a continuous outcome and predictors while sharply reducing the influence of outliers and leverage points. Unlike OLS, which is highly sensitive to extreme observations, robust methods assign down-weighted influence to atypical data points, producing coefficient estimates that remain stable even when a fraction of the data is contaminated or non-normally distributed.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+17 more
When to use it
Use robust regression when the residuals from OLS are heavy-tailed or contain clear outliers that are not data-entry errors but cannot ethically be deleted. It is especially well-suited when outliers are expected on theoretical grounds (e.g., financial returns, survey incomes, biological assays) or when preliminary diagnostics (Cook's D, leverage plots) reveal influential observations. Robust regression assumes the majority of the data follows the assumed linear model — it does not handle gross misspecification of the functional form, nor does it replace proper data cleaning. Do not use it when all observations are trustworthy and normally distributed errors are plausible; OLS then has higher efficiency. For binary, count, or ordinal outcomes, use the appropriate GLM rather than robust linear regression.
Strengths & limitations
- Resists distortion of coefficient estimates and standard errors caused by outliers and leverage points.
- Maintains reasonable efficiency (up to 95% of OLS) when errors are truly Gaussian, depending on the tuning constant.
- MM-estimation achieves a high breakdown point (up to 50%) while retaining high asymptotic efficiency — the gold standard among robust methods.
- Provides a natural diagnostic: the final observation weights flag potential outliers automatically.
- Applicable to the same broad set of linear model structures as OLS (multiple predictors, interactions, polynomials).
- Computationally more intensive than OLS; convergence depends on good starting values (typically obtained via least trimmed squares).
- Inference is asymptotically valid but can be unreliable in very small samples (n < 30).
- Does not address non-linearity or model misspecification — only outlier contamination of otherwise linear relationships.
- Choice of loss function (Huber, bisquare, Hampel) and tuning constants affects results; no single universally best choice exists.
- Less widely implemented in standard software than OLS, and output formats vary across packages.
Frequently asked
How does robust regression differ from OLS?
OLS minimises the sum of squared residuals, making it highly sensitive to outliers. Robust regression minimises a bounded or slower-growing loss function, so extreme observations receive lower weight and have far less influence on the fitted line.
What is the breakdown point?
The breakdown point is the maximum fraction of contaminated data an estimator can tolerate before its estimates become unbounded. OLS has a breakdown point of 0 (a single outlier can destroy it); MM-estimators can achieve up to 50%.
Should I use Huber or bisquare (Tukey) weights?
Huber weights remain linear beyond the threshold, so outliers still exert some influence. Bisquare (Tukey) weights drop to zero beyond the tuning constant, providing stronger resistance. Bisquare is generally preferred when outliers are suspected to be truly anomalous; Huber is milder and may be preferred when atypical points could carry real information.
Can I use robust regression for non-continuous outcomes?
Standard robust regression targets a continuous outcome. For binary, count, or ordinal outcomes, use a robust GLM or quasi-likelihood estimator designed for that outcome type.
Does robust regression eliminate the need for residual diagnostics?
No. Observation weights from the final iteration serve as an outlier diagnostic, but you should still examine residual plots, assess model fit, and investigate heavily down-weighted observations to understand their subject-matter relevance.
Sources
- Huber, P. J. (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1), 73–101. DOI: 10.1214/aoms/1177703732 ↗
- Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., & Stahel, W. A. (1986). Robust Statistics: The Approach Based on Influence Functions. Wiley. ISBN: 978-0471735779
How to cite this page
ScholarGate. (2026, June 3). Robust Regression. ScholarGate. https://scholargate.app/en/statistics/robust-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Lasso RegressionMachine learning↔ compare
- Least Trimmed SquaresStatistics↔ compare
- OLS RegressionEconometrics↔ compare
- Quantile RegressionEconometrics↔ compare
- Ridge RegressionMachine learning↔ compare
- Weighted Least SquaresStatistics↔ compare