Multiple Linear Regression
Multiple Linear Regression (Ordinary Least Squares) · Also known as: MLR, OLS regression, multiple regression, linear regression with multiple predictors, multivariate linear regression
Multiple linear regression (MLR) is a parametric regression model that expresses a continuous outcome as a weighted linear combination of two or more predictor variables plus a random error term. The unknown weights (regression coefficients) are estimated by ordinary least squares (OLS), which minimises the sum of squared residuals. The method traces to Francis Galton's 1886 work on hereditary stature and was placed on firm mathematical footing by Karl Pearson; Draper and Smith's 1966 textbook established it as the standard framework for applied regression.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+11 more
When to use it
Use multiple linear regression when the outcome variable is continuous, the relationship between predictors and outcome is approximately linear, observations are independent, and you have two or more predictors whose simultaneous effects you wish to quantify. Five classical assumptions must hold: (1) linearity — check with partial-regression plots; (2) independence of errors — check with Durbin–Watson if data are time-ordered; (3) homoscedasticity — check with residual-versus-fitted plots and Breusch–Pagan test; (4) approximate normality of residuals — check with a Q-Q plot or Shapiro–Wilk test; (5) absence of perfect multicollinearity — check with variance inflation factors (VIF < 10 as a common rule of thumb). MLR is not appropriate for binary or count outcomes (use logistic or Poisson regression), for strongly nonlinear relationships without transformation, or when the number of predictors approaches or exceeds the sample size.
Strengths & limitations
- Produces directly interpretable coefficients: each β̂_j quantifies the expected change in the outcome per unit increase in predictor j, holding all other predictors constant.
- Provides a complete inferential framework — coefficient t-tests, omnibus F-test, confidence intervals — all within a single model.
- The OLS estimator is analytically exact and computationally efficient even for large datasets.
- Highly flexible: categorical predictors can be incorporated as dummy variables, and interactions or polynomial terms can be added without changing the underlying linear algebra.
- Extremely well-documented: a century of statistical theory supports its properties, diagnostics, and extensions.
- The model is strictly linear in parameters; capturing genuine nonlinearity requires explicit transformations or interaction terms.
- Performance degrades when predictors are highly correlated (multicollinearity), inflating standard errors and making coefficient estimates unstable.
- Outliers and influential observations can distort OLS estimates considerably, because the squared loss function heavily penalises large residuals.
- A large number of predictors relative to sample size leads to overfitting; penalised alternatives (ridge, lasso) are preferred in that regime.
- Violations of homoscedasticity or normality of residuals invalidate standard inference in small samples.
Frequently asked
How many observations do I need relative to the number of predictors?
A widely cited rule of thumb is at least 10–20 observations per predictor for reliable coefficient estimates and valid inference. With fewer observations than predictors, the design matrix is rank-deficient and OLS has no unique solution; penalised regression (ridge, lasso) is required instead.
What is the difference between R² and adjusted R²?
R² measures the proportion of outcome variance explained by the model and always increases when any predictor is added, even an irrelevant one. Adjusted R² penalises each additional predictor by accounting for degrees of freedom, so it can decrease if a new predictor does not contribute meaningfully. Use adjusted R² when comparing models with different numbers of predictors.
My VIF values are above 5 — should I be concerned about multicollinearity?
A VIF above 5 (or 10, depending on the field's convention) indicates that the predictor's variance is inflated due to correlation with other predictors. This does not bias the coefficients but inflates their standard errors, making it harder to detect individual effects. Remedies include centring variables, removing redundant predictors, combining correlated predictors (e.g., via PCA), or switching to ridge regression.
Can I use multiple linear regression with categorical predictors?
Yes. A categorical variable with k levels is encoded as k − 1 dummy variables (binary 0/1 indicators), with one level serving as the reference category. The regression coefficient for each dummy variable then represents the mean difference between that level and the reference, holding all other predictors constant. This is exactly the parameterisation used in ANCOVA and generalised linear models.
Sources
- Galton, F. (1886). Regression towards mediocrity in hereditary stature. Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246–263. DOI: 10.2307/2841583 ↗
- Pearson, K., & Lee, A. (1908). On the generalised probable error in multiple normal correlation. Biometrika, 6(1), 59–68. DOI: 10.1093/biomet/6.1.59 ↗
- Draper, N. R., & Smith, H. (1966). Applied Regression Analysis (1st ed.). John Wiley & Sons. ISBN: 9780471221708
- Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to Linear Regression Analysis (5th ed.). John Wiley & Sons. ISBN: 9780470542811
How to cite this page
ScholarGate. (2026, June 3). Multiple Linear Regression (Ordinary Least Squares). ScholarGate. https://scholargate.app/en/statistics/multiple-linear-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- ANCOVAStatistics↔ compare
- Lasso RegressionMachine learning↔ compare
- Logistic RegressionResearch Statistics↔ compare
- One-way ANOVAStatistics↔ compare
- Polynomial RegressionStatistics↔ compare
- Ridge RegressionMachine learning↔ compare
- Simple Linear RegressionStatistics↔ compare
- Stepwise RegressionStatistics↔ compare