Simple Linear Regression
Simple Linear Regression (OLS) · Also known as: SLR, ordinary least squares regression, OLS regression, bivariate regression, least squares regression
Simple linear regression is the foundational parametric method for modelling a straight-line relationship between one continuous predictor and one continuous outcome, estimating the slope and intercept by ordinary least squares (OLS). The least squares principle was first published by Adrien-Marie Legendre in 1805, and Francis Galton introduced the concept of regression to the mean in 1886, coining the term that names the entire family of methods.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use simple linear regression when you have one continuous predictor and one continuous outcome and want to quantify the direction and magnitude of a linear association, or to predict new outcome values. Four core assumptions must hold: (1) linearity — the true relationship between X and Y is linear; (2) independence — observations are independent of each other; (3) homoscedasticity — the error variance is constant across all values of X; (4) normality — the errors are normally distributed (most important in small samples and for inference). Check linearity with a scatterplot before fitting, and verify the remaining three assumptions via residual diagnostics after fitting. When the predictor is categorical with two levels, simple linear regression is algebraically equivalent to the independent t-test. When more than one predictor is involved, extend to multiple linear regression.
Strengths & limitations
- Provides an explicit, interpretable equation linking predictor to outcome, with slope and intercept that carry direct scientific meaning.
- Closed-form OLS estimates require no iterative optimisation and are computationally trivial even for large datasets.
- The Gauss-Markov theorem guarantees that OLS estimates are the minimum-variance unbiased estimators among all linear estimators when assumptions hold.
- Produces confidence intervals and prediction intervals for both parameters and new observations, supporting rigorous uncertainty quantification.
- Serves as the conceptual foundation for all regression extensions — multiple regression, generalised linear models, mixed models — making it essential pedagogical groundwork.
- Restricted to a single predictor; cannot account for confounding or control for other variables simultaneously.
- The linear assumption fails when the true relationship is curved; misapplication produces biased estimates and poor predictions.
- Sensitive to influential observations and outliers, which can substantially shift the estimated slope.
- Assumes homoscedasticity; when variance increases with X (e.g., income data), OLS standard errors are incorrect and weighted least squares or transformation is needed.
- Extrapolating predictions beyond the observed range of X is unreliable and can produce nonsensical results.
Frequently asked
What is the difference between correlation and simple linear regression?
Pearson correlation measures the strength and direction of the linear association between two variables without distinguishing predictor from outcome; it is symmetric. Simple linear regression assigns one variable as the predictor and the other as the outcome, estimates a slope and intercept, and produces predictions. In simple regression, R² equals the square of the Pearson correlation coefficient, linking the two, but regression provides additional inferential machinery — confidence intervals for the slope, prediction intervals for new observations — that correlation alone does not.
How do I check whether the linear model is appropriate?
Start with a scatterplot of Y versus X before fitting; a systematic curve suggests nonlinearity. After fitting, examine the residual-versus-fitted plot — a random horizontal scatter indicates linearity and homoscedasticity; a funnel shape suggests heteroscedasticity; a curve suggests a nonlinear relationship. A normal Q-Q plot of residuals checks the normality assumption. For influential points, inspect Cook's distance values.
Does a high R² mean the model is good?
Not necessarily. R² measures only how much variance is explained, not whether the model is correctly specified. A high R² can coexist with systematic patterns in the residuals (indicating model misspecification), while a low R² can be perfectly acceptable in noisy fields such as psychology or economics where human behaviour is inherently variable. Always inspect residual diagnostics and report the slope estimate with its confidence interval, not R² alone.
What should I do if my data violate the homoscedasticity assumption?
First confirm heteroscedasticity visually (funnel pattern in residuals vs. fitted plot) or with a formal test (e.g., Breusch-Pagan). Common remedies include applying a variance-stabilising transformation to Y (e.g., log or square root), using weighted least squares with weights proportional to the inverse of the error variance, or computing heteroscedasticity-consistent (HC) standard errors (White standard errors) without altering the point estimates.
Sources
- Legendre, A. M. (1805). Nouvelles méthodes pour la détermination des orbites des comètes. Firmin Didot, Paris. [Appendix: Sur la méthode des moindres quarrés, pp. 72–80] link ↗
- Galton, F. (1886). Regression towards mediocrity in hereditary stature. Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246–263. DOI: 10.2307/2841583 ↗
- Montgomery, D. C., Peck, E. A., & Vining, G. G. (2021). Introduction to Linear Regression Analysis (6th ed.). Wiley. ISBN: 978-1119578727
How to cite this page
ScholarGate. (2026, June 3). Simple Linear Regression (OLS). ScholarGate. https://scholargate.app/en/statistics/simple-linear-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Independent t-testStatistics↔ compare
- Logistic RegressionResearch Statistics↔ compare
- Multiple Linear RegressionStatistics↔ compare
- One-way ANOVAStatistics↔ compare
- Pearson CorrelationStatistics↔ compare
- Polynomial RegressionStatistics↔ compare
- Ridge RegressionMachine learning↔ compare