Ridge Regression
Ridge Regression (L2-Regularized Linear Regression) · Also known as: Ridge Regresyonu, ridge regresyonu, L2-regularized regression, Tikhonov regularization
Ridge Regression is an L2-regularized linear regression method, introduced by Arthur Hoerl and Robert Kennard in 1970, that reduces multicollinearity by adding a penalty on the size of the coefficients. It shrinks coefficients toward zero without setting any of them exactly to zero, producing more stable estimates when predictors are highly correlated.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+22 more
When to use it
Use Ridge Regression for prediction with continuous, binary, or count predictors when multicollinearity is present — for example when a variance inflation factor exceeds 10 — and you want to keep all variables rather than select among them. It assumes a roughly linear relationship and at least about 30 observations. Features must be standardized first. When samples are small and collinearity is mild, plain linear regression is more interpretable; when variable selection is the goal, LASSO or Elastic Net is preferable.
Strengths & limitations
- Reduces multicollinearity and stabilizes coefficient estimates by adding an L2 penalty.
- Keeps every predictor in the model — coefficients shrink but none are dropped to zero.
- Lowers variance at the cost of a small, controlled increase in bias, often improving prediction.
- Has a single, easily tuned hyperparameter (alpha) chosen by cross-validation.
- Works with continuous, binary, and count predictors without requiring normally distributed data.
- Performs no variable selection; it cannot zero out irrelevant predictors the way LASSO can.
- Requires features to be standardized, otherwise the penalty unfairly favours large-scale variables.
- Assumes a linear relationship between predictors and outcome.
- On small samples (n below about 30) tuning alpha by cross-validation is unreliable and plain OLS may suffice.
- When collinearity is not actually a problem, the shrinkage is unnecessary and reduces interpretability.
Frequently asked
How is Ridge different from LASSO?
Both add a penalty to shrink coefficients, but Ridge uses an L2 penalty that shrinks coefficients toward zero without ever reaching it, so all predictors stay in the model. LASSO uses an L1 penalty that can set coefficients exactly to zero, performing variable selection. Choose Ridge to stabilize correlated predictors and LASSO when you need a sparse model.
Do I need to standardize my features?
Yes. Standardizing features is required, because the L2 penalty acts on the size of the coefficients; without standardization the penalty would unfairly shrink variables measured on larger scales.
How do I choose the penalty strength alpha?
Choose alpha by cross-validation: scan a range of values and pick the one that minimizes the cross-validated mean squared error. Plotting the CV MSE curve against alpha shows where the optimum lies.
When should I not use Ridge?
If there is no real multicollinearity, Ridge adds unnecessary shrinkage and a plain linear regression is more interpretable. With very small samples (n below about 30), tuning alpha is unreliable, and if you need to drop irrelevant variables you should use LASSO or Elastic Net instead.
Sources
- Hoerl, A.E. & Kennard, R.W. (1970). Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics, 12(1), 55–67. DOI: 10.1080/00401706.1970.10488634 ↗
How to cite this page
ScholarGate. (2026, June 1). Ridge Regression (L2-Regularized Linear Regression). ScholarGate. https://scholargate.app/en/machine-learning/ridge-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Elastic NetMachine learning↔ compare
- Lasso RegressionMachine learning↔ compare
- Logistic RegressionResearch Statistics↔ compare
- Principal Component AnalysisMachine learning↔ compare