M-Estimators (Robust Regression)
Also known as: m-estimation, huber regression, robust m-regression, M-Tahmin Ediciler
M-estimators are a robust generalisation of maximum likelihood estimation, formalised in the work of Peter J. Huber (Huber & Ronchetti, 2009). Instead of squaring every residual, they apply a bounded loss function so that large residuals from outliers are down-weighted rather than allowed to dominate the fit.
Key highlights
- Down-weights outliers in the response without requiring you to delete observations by hand.
- Generalises maximum likelihood estimation, with interpretable coefficients on the same scale as ordinary regression.
- Flexible loss functions (Huber, Bisquare, Andrews) let you tune how strongly extreme residuals are suppressed.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use M-estimators when you fit a linear relationship to a continuous outcome but the data contain outliers in the response that you do not want to discard manually. They work best when the residuals are roughly symmetric and the share of outliers does not exceed about 25%, with a reasonable sample size (at least about 30 observations). They are not appropriate when contamination comes from high-leverage points in the predictor space — there a high-breakdown method such as an MM-estimator is needed — or when the outlier fraction is so large that the breakdown point is exceeded.
Strengths & limitations
- Down-weights outliers in the response without requiring you to delete observations by hand.
- Generalises maximum likelihood estimation, with interpretable coefficients on the same scale as ordinary regression.
- Flexible loss functions (Huber, Bisquare, Andrews) let you tune how strongly extreme residuals are suppressed.
- Misleading in the presence of leverage points in the predictor space, where a high-breakdown method is required instead.
- The breakdown point is exceeded once the outlier ratio rises above about 25%.
- Assumes the residuals are symmetrically distributed; skewed errors undermine the robust loss.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How is an M-estimator different from OLS?
OLS minimises the sum of squared residuals, so a single outlier can dominate the fit. An M-estimator minimises a bounded loss function instead, smoothly down-weighting large residuals so the line follows the majority of the data.
What do the weight functions (Huber, Bisquare, Andrews) do?
They define the robust loss ρ and therefore how residuals are penalised. The Huber function down-weights large residuals gently, while Bisquare and Andrews can give very extreme residuals essentially zero weight. The choice controls the trade-off between efficiency and resistance to outliers.
When should I use an MM-estimator instead?
When the contamination comes from high-leverage points in the predictor space. M-estimators can be misled there because they do not have a high breakdown point; an MM-estimator is designed to remain reliable in that situation.
How many outliers can an M-estimator tolerate?
Roughly up to a quarter of the observations. Once the outlier ratio exceeds about 25% the breakdown point is exceeded and the estimate becomes unreliable; a method such as least trimmed squares is then more appropriate.
Sources
- 1.Huber, P. J., & Ronchetti, E. M. (2009). Robust Statistics (2nd ed.). Wiley.
- 2.Maronna, R. A., Martin, R. D., Yohai, V. J., & Salibián-Barrera, M. (2019). Robust Statistics: Theory and Methods (with R) (2nd ed.). Wiley.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 1). M-Estimator. ScholarGate. https://scholargate.app/statistics/m-estimator