Robust Multiple Linear Regression
Also known as: robust MLR, M-estimator regression, resistant multiple regression, robust OLS
Robust multiple linear regression estimates the linear relationship between a continuous outcome and several predictors while being resistant to outliers and violations of the normality assumption. Instead of minimising the sum of squared residuals, it uses a bounded loss function — most commonly Huber's or Tukey's bisquare — so that extreme observations receive limited influence on the estimated coefficients.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+1 more
When to use it
Use robust multiple linear regression when you have a continuous outcome and multiple predictors but suspect the data contain outliers, heavy-tailed errors, or moderate departures from normality that would distort OLS estimates. It is particularly valuable in exploratory work, observational studies, or fields such as economics, environmental science, and engineering where influential anomalies are common. Do not use it as a routine substitute for OLS without first inspecting the data: if the errors are genuinely normal and there are no outliers, OLS is more efficient. Also avoid it when high-leverage points are meaningful, correct observations that should not be down-weighted.
Strengths & limitations
- Resistant to outliers: a small fraction of contaminated observations has limited effect on the estimated coefficients.
- No assumption of normally distributed errors; valid under heavy-tailed or slightly asymmetric error distributions.
- Retains the familiar multiple regression framework with interpretable linear coefficients.
- MM-estimators achieve both high breakdown point (up to 50% contamination) and high asymptotic efficiency near normality.
- Automatically flags influential observations through the estimated weights, aiding diagnostic insight.
- Computationally more demanding than OLS, requiring iterative algorithms that may not always converge.
- Less efficient than OLS when errors are truly normally distributed with no outliers.
- Down-weighting legitimate high-leverage observations (e.g., extreme but valid experimental conditions) can bias estimates toward the majority of the data.
- Inference (standard errors, confidence intervals) relies on asymptotic approximations that may be unreliable in small samples.
Frequently asked
How does robust regression differ from OLS?
OLS minimises the sum of squared residuals, giving equal influence to every observation and very high influence to outliers. Robust regression uses a bounded loss function and iteratively down-weights extreme residuals, so outliers have limited effect on the estimated coefficients.
What is a breakdown point?
The breakdown point is the largest fraction of contaminated observations an estimator can tolerate before its estimates become arbitrarily bad. OLS has a breakdown point of 0% (a single outlier can ruin it); MM-estimators can achieve up to 50%.
Should I choose Huber or Tukey bisquare loss?
Huber's loss is quadratic near zero and linear in the tails, giving moderate resistance. Tukey's bisquare completely zeroes out the influence of observations beyond a threshold, offering stronger resistance to gross outliers but potentially ignoring valid extreme data. For most applied work, MM-estimators (which use bisquare in the efficient step) are a good default.
Can I use robust regression with categorical predictors?
Yes. Categorical predictors are dummy-coded exactly as in OLS. Robust regression operates on the residuals from the fitted linear surface, so the encoding of predictors is unchanged.
Is robust regression appropriate for small samples?
Robust methods rely on asymptotic theory, and their standard errors are less reliable in small samples (roughly n < 50). Bootstrap confidence intervals are preferable when n is small and outliers are suspected.
Sources
- Huber, P. J. (1964). Robust estimation of a location parameter. Annals of Mathematical Statistics, 35(1), 73–101. DOI: 10.1214/aoms/1177703732 ↗
- Maronna, R. A., Martin, R. D., & Yohai, V. J. (2006). Robust Statistics: Theory and Methods. Wiley. ISBN: 978-0470010921
How to cite this page
ScholarGate. (2026, June 3). Robust Multiple Linear Regression. ScholarGate. https://scholargate.app/en/statistics/robust-multiple-linear-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Lasso RegressionMachine learning↔ compare
- Multiple Linear RegressionStatistics↔ compare
- OLS RegressionEconometrics↔ compare
- Quantile RegressionEconometrics↔ compare
- Ridge RegressionMachine learning↔ compare
- Robust RegressionStatistics↔ compare