Lasso Regression
Least Absolute Shrinkage and Selection Operator (LASSO) · Also known as: LASSO Regresyonu, lasso, L1-regularized regression, L1 regularization
Lasso regression, introduced by Robert Tibshirani in 1996, is a linear regression method that adds an L1 penalty to the loss so that it shrinks coefficients and performs variable selection at the same time, producing a sparse model. By driving some coefficients exactly to zero it keeps only the predictors that matter.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+15 more
When to use it
A good fit when you want a sparse, interpretable linear model for prediction or explanation from many continuous, binary, or count predictors, and at least about 30 observations. It is especially useful when multicollinearity is present (for example VIF above 10) and you want the method itself to choose which variables to keep. It assumes a roughly linear relationship and requires the features to be standardised. Below about 30 observations the variable selection becomes unstable and ridge regression is safer; when the number of predictors exceeds the sample size, elastic net is preferable.
Strengths & limitations
- Performs variable selection and coefficient shrinkage in a single fit, yielding a sparse, interpretable model.
- Drives weak predictors exactly to zero, which simplifies the model and reduces overfitting.
- A natural response to multicollinearity (high VIF) without manually dropping variables.
- Does not require normally distributed data and handles continuous, binary, and count predictors.
- Only one tuning parameter (lambda), which is straightforward to set by cross-validation.
- Assumes a linear relationship between predictors and outcome.
- Features must be standardised first or the penalty treats variables on different scales unfairly.
- On small samples (n below about 30) the selected variables are unstable and not reproducible.
- When the number of predictors exceeds the sample size it can select at most n variables, so elastic net is preferred.
Frequently asked
How is lambda chosen?
Lambda controls how strongly coefficients are penalised. It is selected by cross-validation: a range of values is tried and the one minimising the cross-validated mean squared error is kept.
Why must I standardise the features?
The single L1 penalty applies equally to every coefficient, so predictors measured on larger scales would otherwise be penalised less. Standardising the features first makes the penalty fair across variables.
How does Lasso differ from ridge regression?
Lasso uses an L1 penalty that can set coefficients exactly to zero, performing variable selection, while ridge uses an L2 penalty that shrinks coefficients but does not zero them out. On small samples or when selection is unstable, ridge is the safer choice.
What should I check after fitting?
Inspect the cross-validation MSE curve to confirm the chosen lambda, review which variables were kept versus set to zero, and examine residual plots for normality and homogeneity.
Sources
- Tibshirani, R. (1996). Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society: Series B, 58(1), 267–288. DOI: 10.1111/j.2517-6161.1996.tb02080.x ↗
How to cite this page
ScholarGate. (2026, June 1). Least Absolute Shrinkage and Selection Operator (LASSO). ScholarGate. https://scholargate.app/en/machine-learning/lasso-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Elastic NetMachine learning↔ compare
- Logistic RegressionResearch Statistics↔ compare
- Principal Component AnalysisMachine learning↔ compare
- Ridge RegressionMachine learning↔ compare