Linear Regression (ML)
Linear Regression as a Machine Learning Model · Also known as: ordinary least squares regression, OLS, least squares regression, multiple linear regression
Linear regression fits a straight-line relationship between one or more input features and a continuous numeric outcome by minimising the sum of squared prediction errors. As a machine-learning model it is trained on labeled examples and evaluated on held-out data, making it the simplest supervised learning baseline for any regression task.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use linear regression when the outcome is continuous, the expected relationship between predictors and outcome is approximately linear, and interpretable coefficients are needed. It is the right starting baseline before trying more complex models, and a good final model when its assumptions hold. Do not use it for binary or categorical outcomes (use logistic regression instead), when strong nonlinear patterns or interactions dominate, or when the number of predictors exceeds the sample size without regularisation.
Strengths & limitations
- Coefficients are directly interpretable: each value is a marginal effect in the outcome's units.
- Computationally trivial — fits almost instantly even on large datasets.
- Provides standard errors and p-values, supporting formal hypothesis testing.
- Serves as the indispensable baseline against which more complex models should be compared.
- Works well and is fully justified when the linearity assumption holds.
- Highly compatible with regularisation extensions (Ridge, Lasso, Elastic Net) when predictors are many.
- Assumes a linear relationship; misses curves, thresholds, and interactions unless they are manually encoded.
- Sensitive to outliers because squaring errors amplifies the influence of extreme points.
- Multicollinearity among predictors inflates standard errors and destabilises coefficient estimates.
- Requires that residuals be approximately normally distributed and homoscedastic for valid inference.
Frequently asked
How is linear regression different from logistic regression?
Linear regression predicts a continuous outcome directly. Logistic regression models the log-odds of a binary outcome and outputs probabilities bounded between 0 and 1. Using linear regression for a binary outcome can produce predictions outside [0, 1] and violates the constant-variance assumption.
Do I need to standardise my features?
Standardisation is not required for predictions, but it makes coefficients directly comparable when features have very different scales. It is especially recommended when using regularised variants such as Ridge or Lasso.
How do I handle many predictors relative to sample size?
Switch to regularised linear regression (Ridge for shrinkage, Lasso for sparsity, Elastic Net for both). OLS becomes unreliable or undefined when the number of predictors approaches or exceeds the number of observations.
What residual plots should I inspect?
At minimum: (1) residuals vs. fitted values to check for patterns suggesting nonlinearity or heteroscedasticity, and (2) a Q-Q plot of residuals to assess normality. A scale-location plot and leverage/influence statistics (Cook's distance) are also informative.
Is a high R-squared enough to declare a good model?
No. R-squared measures in-sample fit and can be inflated by irrelevant predictors. Always report cross-validated or test-set RMSE alongside R-squared, and inspect residual diagnostics to confirm the model's assumptions are satisfied.
Sources
- Hastie, T., Tibshirani, R. & Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd ed., Ch. 3). Springer. ISBN: 978-0-387-84858-7
- James, G., Witten, D., Hastie, T. & Tibshirani, R. (2013). An Introduction to Statistical Learning (Ch. 3). Springer. ISBN: 978-1-4614-7138-7
How to cite this page
ScholarGate. (2026, June 3). Linear Regression as a Machine Learning Model. ScholarGate. https://scholargate.app/en/machine-learning/linear-regression-ml
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Decision TreeMachine learning↔ compare
- Gradient BoostingMachine learning↔ compare
- Logistic regression (ML)Machine learning↔ compare
- Random ForestMachine learning↔ compare
- Regularized linear regressionMachine learning↔ compare