Mean Squared Error (MSE)
Also known as: MSE, L2 error, quadratic error
Mean Squared Error is the foundational loss function for regression models, measuring the average squared deviation between predictions and observations. Originating from Gauss and Legendre's method of least squares (1805-1809), MSE is the basis for ordinary least squares regression and remains central to modern machine learning optimization.
Key highlights
- Smooth and differentiable everywhere, enabling efficient optimization
- Mathematically convenient for closed-form solutions and variance calculations
- Heavy penalty on large errors matches many real-world cost structures
- Foundation for rigorous statistical inference (confidence intervals, hypothesis tests)
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
MSE is the default choice when fitting regression models via gradient descent or closed-form solutions; it is the loss function that makes least-squares estimation tractable. Use MSE when: (1) you want to heavily penalize large errors, (2) you are building models where the optimization landscape matters, or (3) you need to compute variance and confidence intervals from residual statistics. Be cautious with MSE if your data contains outliers, as their squared errors will dominate.
Strengths & limitations
- Smooth and differentiable everywhere, enabling efficient optimization
- Mathematically convenient for closed-form solutions and variance calculations
- Heavy penalty on large errors matches many real-world cost structures
- Foundation for rigorous statistical inference (confidence intervals, hypothesis tests)
- Highly sensitive to outliers; single extreme values dominate the metric
- Expressed in squared units, making direct interpretation difficult
- Does not directly reflect prediction accuracy in original scale
- Asymmetric treatment of under- and over-predictions (both squared equally, but interpretation varies by domain)
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What is the difference between MSE and RMSE?
MSE is the mean of squared errors (in squared units). RMSE is the square root of MSE, returning the metric to original units. They rank models identically, but RMSE is more interpretable while MSE is easier to optimize.
Why is MSE preferred for optimization over MAE?
MSE is smooth and differentiable everywhere, allowing efficient gradient descent. MAE has a non-smooth kink at zero, complicating gradient-based optimization. However, specialized solvers can handle MAE.
Can I use MSE as a loss function for classification?
Technically yes, but it is not recommended. MSE treats classification as regression and can lead to poor probability estimates. Use cross-entropy loss (logistic loss) for binary classification or categorical cross-entropy for multiclass.
How does MSE relate to variance?
MSE can be decomposed as: MSE = Bias² + Variance + Irreducible Error. This decomposition is fundamental to understanding the bias-variance tradeoff in model selection.
Is high MSE always bad?
Not necessarily. If your target variable has high variance, even perfect predictions will have nonzero MSE due to irreducible error. Always compare MSE to baseline models and normalize by target variance (via R²) for context.
Sources
- 1.Gauss, C. F. (1809). Theoria Motus Corporum Coelestium in Sectionibus Conicis Solem Ambientium. Hamburg: Perthes and Besser.
- 2.Legendre, A. M. (1805). Nouvelles méthodes pour la détermination des orbites des comètes. Paris: F. Didot.
- 3.Goodman, L. A. (1960). On the exact variance of products. Journal of the American Statistical Association, 55(292), 708-713.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Mean Squared Error. ScholarGate. https://scholargate.app/model-evaluation/mean-squared-error