Huber Regression
Also known as: Huber M-estimator, Huber loss regression, robust regression, Huber Regresyonu
Huber regression is a robust linear regression method, introduced by Peter J. Huber in 1964, that resists the influence of outliers by treating small and large residuals differently. It applies a squared (OLS-like) loss to small residuals and a milder absolute-value loss to large ones, so extreme observations cannot dominate the fit.
Key highlights
- Resistant to outliers in the outcome: large residuals receive only a bounded, linear penalty so they cannot dominate the fit.
- Behaves like efficient OLS in the clean part of the data, combining good efficiency with robustness.
- Produces interpretable linear coefficients, so results read like a familiar regression.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use Huber regression when the outcome is continuous, the relationship is approximately linear, and you suspect the data contains outliers that would distort an ordinary least squares fit. It works best with a reasonable sample (at least about 30 observations) and assumes the underlying relationship is linear. It is intended for cross-sectional or longitudinal continuous data where most points follow the model but a minority are contaminated. It is less suitable when high-leverage points are present or when the outlier proportion is large.
Strengths & limitations
- Resistant to outliers in the outcome: large residuals receive only a bounded, linear penalty so they cannot dominate the fit.
- Behaves like efficient OLS in the clean part of the data, combining good efficiency with robustness.
- Produces interpretable linear coefficients, so results read like a familiar regression.
- Breaks down once the outlier proportion exceeds about 25%, where a high-breakdown estimator is needed instead.
- Vulnerable to high-leverage points (outliers in the predictor space), where a bounded-influence estimator such as an MM-estimator is preferable.
- Assumes a linear relationship and needs a reasonable sample size (at least about 30 observations).
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How does Huber regression differ from OLS?
OLS squares every residual, so far-off points have a large, unbounded influence. Huber regression uses a squared loss only for small residuals and switches to a linear loss for large ones, capping the influence of any single outlier while staying efficient on the clean data.
What is the threshold δ (epsilon) in Huber regression?
δ is the residual size at which the loss switches from quadratic to linear. Smaller δ makes the method more robust to outliers but less efficient; larger δ behaves more like OLS. It governs the trade-off between robustness and efficiency.
How many outliers can Huber regression tolerate?
Huber regression is reliable up to roughly 25% outliers. Beyond that it reaches its breakdown point, and a high-breakdown method such as least trimmed squares is more appropriate.
What about high-leverage points?
Huber regression bounds the influence of outliers in the outcome but not of outliers in the predictor space (high-leverage points). When such points are present, a bounded-influence estimator such as an MM-estimator is the better choice.
Sources
- 1.Huber, P. J. (1964). Robust Estimation of a Location Parameter. Annals of Mathematical Statistics, 35(1), 73-101.
- 2.Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., & Stahel, W. A. (1986). Robust Statistics: The Approach Based on Influence Functions. Wiley.ISBN 978-0471735779
You have read it. What now?
Cite this page
ScholarGate. (2026, June 1). Huber Regression. ScholarGate. https://scholargate.app/statistics/huber-regression