Robust Regression
Also known as: M-estimation regression, robust linear regression, outlier-resistant regression, MM-estimation
Robust regression estimates the linear relationship between a continuous outcome and predictors while sharply reducing the influence of outliers and leverage points. Unlike OLS, which is highly sensitive to extreme observations, robust methods assign down-weighted influence to atypical data points, producing coefficient estimates that remain stable even when a fraction of the data is contaminated or non-normally distributed.
Key highlights
- Resists distortion of coefficient estimates and standard errors caused by outliers and leverage points.
- Maintains reasonable efficiency (up to 95% of OLS) when errors are truly Gaussian, depending on the tuning constant.
- MM-estimation achieves a high breakdown point (up to 50%) while retaining high asymptotic efficiency — the gold standard among robust methods.
- Provides a natural diagnostic: the final observation weights flag potential outliers automatically.
- Applicable to the same broad set of linear model structures as OLS (multiple predictors, interactions, polynomials).
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use robust regression when the residuals from OLS are heavy-tailed or contain clear outliers that are not data-entry errors but cannot ethically be deleted. It is especially well-suited when outliers are expected on theoretical grounds (e.g., financial returns, survey incomes, biological assays) or when preliminary diagnostics (Cook's D, leverage plots) reveal influential observations. Robust regression assumes the majority of the data follows the assumed linear model — it does not handle gross misspecification of the functional form, nor does it replace proper data cleaning. Do not use it when all observations are trustworthy and normally distributed errors are plausible; OLS then has higher efficiency. For binary, count, or ordinal outcomes, use the appropriate GLM rather than robust linear regression.
Strengths & limitations
- Resists distortion of coefficient estimates and standard errors caused by outliers and leverage points.
- Maintains reasonable efficiency (up to 95% of OLS) when errors are truly Gaussian, depending on the tuning constant.
- MM-estimation achieves a high breakdown point (up to 50%) while retaining high asymptotic efficiency — the gold standard among robust methods.
- Provides a natural diagnostic: the final observation weights flag potential outliers automatically.
- Applicable to the same broad set of linear model structures as OLS (multiple predictors, interactions, polynomials).
- Computationally more intensive than OLS; convergence depends on good starting values (typically obtained via least trimmed squares).
- Inference is asymptotically valid but can be unreliable in very small samples (n < 30).
- Does not address non-linearity or model misspecification — only outlier contamination of otherwise linear relationships.
- Choice of loss function (Huber, bisquare, Hampel) and tuning constants affects results; no single universally best choice exists.
- Less widely implemented in standard software than OLS, and output formats vary across packages.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How does robust regression differ from OLS?
OLS minimises the sum of squared residuals, making it highly sensitive to outliers. Robust regression minimises a bounded or slower-growing loss function, so extreme observations receive lower weight and have far less influence on the fitted line.
What is the breakdown point?
The breakdown point is the maximum fraction of contaminated data an estimator can tolerate before its estimates become unbounded. OLS has a breakdown point of 0 (a single outlier can destroy it); MM-estimators can achieve up to 50%.
Should I use Huber or bisquare (Tukey) weights?
Huber weights remain linear beyond the threshold, so outliers still exert some influence. Bisquare (Tukey) weights drop to zero beyond the tuning constant, providing stronger resistance. Bisquare is generally preferred when outliers are suspected to be truly anomalous; Huber is milder and may be preferred when atypical points could carry real information.
Can I use robust regression for non-continuous outcomes?
Standard robust regression targets a continuous outcome. For binary, count, or ordinal outcomes, use a robust GLM or quasi-likelihood estimator designed for that outcome type.
Does robust regression eliminate the need for residual diagnostics?
No. Observation weights from the final iteration serve as an outlier diagnostic, but you should still examine residual plots, assess model fit, and investigate heavily down-weighted observations to understand their subject-matter relevance.
Sources
- 1.Huber, P. J. (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1), 73–101.
- 2.Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., & Stahel, W. A. (1986). Robust Statistics: The Approach Based on Influence Functions. Wiley.ISBN 978-0471735779
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Robust Regression. ScholarGate. https://scholargate.app/statistics/robust-regression