Multiple Regression Analysis
Multiple Linear Regression · Also known as: MLR, multivariate regression, linear regression
Multiple regression analysis is a statistical method for modeling the relationship between a continuous dependent variable and two or more independent variables (predictors). Originating from Gauss's early 19th-century work and formalized by Draper and Smith (1966), it estimates linear equations predicting outcomes from multiple predictors while accounting for confounding relationships, making it indispensable in epidemiology, economics, psychology, and clinical research.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use multiple regression when you have a continuous outcome and multiple predictors to understand their associations while adjusting for confounders. Common applications: predicting patient outcomes from clinical variables, understanding factors driving economic indicators, estimating treatment effect while controlling demographics, modeling educational achievement from student characteristics. Regression is also ideal when sample sizes prevent experimental randomization (observational studies). For categorical outcomes, use logistic or multinomial regression. For clustered data (students in schools), use multilevel regression. For missing data, consider multiple imputation beforehand.
Strengths & limitations
- Powerful for disentangling complex relationships: estimates each predictor's unique effect while controlling others, simulating experimental control in observational data.
- Flexible: accommodates continuous and categorical predictors, interactions, polynomial terms, and can be extended to nonlinear relationships.
- Interpretable: coefficients directly represent unit change in outcome, making results actionable for clinicians and policymakers.
- Prediction: fitted model generates predictions for new observations with confidence intervals, enabling practical applications (risk stratification, forecasting).
- Extensive diagnostics available: standardized residuals, leverage plots, and influence measures help identify and address model violations.
- Assumes linear relationships; misses curvilinear or threshold effects unless explicitly modeled via transformation or polynomial terms.
- Sensitive to multicollinearity: highly correlated predictors produce unstable, difficult-to-interpret coefficients. VIF > 10 problematic; solution includes removing predictors, principal components, or ridge regression.
- Outliers and influential points can disproportionately affect estimates and fit; robust regression methods (Huber, M-estimators) address this.
- Assumes constant variance (homoscedasticity); heteroscedastic data requires weighted least squares or robust standard error adjustments.
- Statistical significance is not causal: cannot distinguish whether association reflects true effect or unmeasured confounding; requires strong theoretical justification.
Frequently asked
When should I standardize predictors, and how does it affect interpretation?
Standardize continuous predictors (z-score: subtract mean, divide by standard deviation) when: (1) predictors are on vastly different scales and you want to compare magnitude of coefficients, or (2) you need to reduce multicollinearity. Standardized coefficients are in units of standard deviations: a coefficient of 0.5 means a 1-SD increase in the predictor predicts a 0.5-SD increase in the outcome. Raw (unstandardized) coefficients retain original units and often aid practical interpretation. Always report both and specify which you report.
What is the difference between R² and adjusted R²?
R² = 1 − (SSE/SS_total) represents the proportion of outcome variance explained by the model. It always increases when predictors are added, even if they have no true effect. Adjusted R² = 1 − [(1 − R²)(n − 1)/(n − p − 1)] penalizes model complexity by degrees of freedom lost (p is number of predictors, n is sample size). For model selection and assessment of true predictive power, use adjusted R²; it decreases if a new predictor adds less information than its cost (one parameter). Report both.
How do I interpret a nonsignificant coefficient when other variables are in the model?
A nonsignificant coefficient (p > 0.05) indicates insufficient evidence that this predictor is associated with the outcome, holding other variables constant. This could mean the predictor has no effect, is confounded by other predictors (multicollinearity), or the sample size is too small. Check the confidence interval and VIF. If confidence interval is wide and VIF is high, multicollinearity is likely. If the predictor is theoretically important, consider reporting it with 90% CI as a sensitivity analysis. Do not drop predictors based solely on p-values; use domain knowledge.
What do I do if residuals are not normally distributed?
Mild non-normality is often acceptable due to robustness of regression with large samples; normality of residuals becomes critical with n < 30. Check Q-Q plot visually and conduct Shapiro-Wilk test. If non-normal, consider: (1) log or square-root transformation of the outcome, (2) adding omitted predictors or interactions, (3) fitting robust regression (Huber, bisquare), or (4) using bootstrap confidence intervals. Extreme outliers must be investigated; remove if due to data entry error, otherwise report results with and without outlier.
Sources
- Draper, N. R., & Smith, H. (1966). Applied Regression Analysis. John Wiley & Sons. link ↗
- Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (1992). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum. link ↗
- Marquardt, D. W. (1980). You should standardize the independent variables in your regression models. Discussion of a paper by G. David Knottnerus. Journal of the American Statistical Association, 75(369), 87–91. link ↗
How to cite this page
ScholarGate. (2026, June 4). Multiple Linear Regression. ScholarGate. https://scholargate.app/en/research-statistics/multiple-regression-analysis
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Analysis of Variance (ANOVA)Research Statistics↔ compare
- Factor AnalysisResearch Statistics↔ compare
- Logistic RegressionResearch Statistics↔ compare
- Structural Equation ModelingResearch Statistics↔ compare