Bayesian LASSO Regression
Also known as: Bayesian LASSO, Bayesian L1 regression, double-exponential prior regression, Laplace prior regression
Bayesian LASSO regression places double-exponential (Laplace) priors on regression coefficients, which is the Bayesian analogue of the classical LASSO penalty. It simultaneously shrinks small coefficients toward zero and performs soft variable selection, all within a coherent posterior inference framework that naturally quantifies parameter uncertainty through credible intervals.
Key highlights
- Provides full posterior distributions for all coefficients, yielding credible intervals that correctly reflect estimation uncertainty after shrinkage.
- Performs automatic shrinkage and soft variable selection simultaneously, without a separate model-selection step.
- The hyperparameter λ can be estimated from the data via a hyperprior, removing the need for cross-validation.
- More robust to moderate multicollinearity than OLS: the Laplace prior provides strong regularization in collinear directions.
- Naturally extends to hierarchical and generalized-linear settings by replacing only the prior on β.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use Bayesian LASSO when you have a continuous outcome, potentially many predictors relative to sample size, and you want automatic shrinkage and soft variable selection together with honest uncertainty quantification via posterior credible intervals. It is particularly valuable when you want to incorporate prior knowledge about the shrinkage strength or when classical LASSO's confidence-interval shortcomings matter for your application. Avoid it when you need exact sparse solutions (posterior mode may not be exactly zero for small coefficients), when you have very many predictors and computational speed is critical (MCMC scales poorly), or when a simple, fast, and transparently interpretable point estimate suffices.
Strengths & limitations
- Provides full posterior distributions for all coefficients, yielding credible intervals that correctly reflect estimation uncertainty after shrinkage.
- Performs automatic shrinkage and soft variable selection simultaneously, without a separate model-selection step.
- The hyperparameter λ can be estimated from the data via a hyperprior, removing the need for cross-validation.
- More robust to moderate multicollinearity than OLS: the Laplace prior provides strong regularization in collinear directions.
- Naturally extends to hierarchical and generalized-linear settings by replacing only the prior on β.
- Does not produce exact zeros in the posterior mean or median; near-zero mass must be interpreted rather than read off directly.
- MCMC computation is substantially slower than coordinate-descent algorithms for classical LASSO, limiting scalability to very high-dimensional problems.
- Results depend on the choice of hyperprior for λ; a poorly specified hyperprior can under- or over-shrink.
- Posterior mixing can be slow when many predictors are correlated, requiring careful MCMC convergence diagnostics.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
Does the Bayesian LASSO set coefficients exactly to zero?
No. Unlike the classical LASSO, the posterior distribution assigns positive mass everywhere, so the posterior mean and median are never exactly zero. Small coefficients are strongly shrunk toward zero but retain residual uncertainty. For hard variable selection, examine posterior credible intervals or use spike-and-slab priors instead.
How does the Bayesian LASSO differ from Bayesian ridge regression?
Bayesian ridge regression places Gaussian (normal) priors on coefficients, producing L2-penalized posterior modes and gentle, uniform shrinkage. The Bayesian LASSO uses Laplace (double-exponential) priors, which are more peaked at zero and heavier-tailed, inducing stronger shrinkage for small coefficients while allowing large coefficients to remain relatively unshrunk — the Bayesian analogue of L1 vs. L2 penalties.
How is the shrinkage parameter λ chosen?
λ can be given a gamma hyperprior and estimated from the data as part of the MCMC — the fully Bayesian approach advocated by Park and Casella (2008). Alternatively, λ can be fixed by empirical Bayes (marginal likelihood maximization) or by cross-validation, mirroring how the classical LASSO tunes its penalty.
When should I prefer classical LASSO or Elastic Net over Bayesian LASSO?
If you need a fast, sparse solution with exact zero coefficients and do not need posterior credible intervals, classical LASSO or Elastic Net computed via coordinate descent are much faster and scale to very high dimensions. Prefer Bayesian LASSO when uncertainty quantification, hierarchical modeling, or prior elicitation is important.
Can Bayesian LASSO be applied to generalized linear models?
Yes. The Laplace prior can be placed on coefficients in any GLM (logistic, Poisson, etc.), and inference proceeds via Metropolis-within-Gibbs or variational approximations. Stan, PyMC, and R packages such as monomvn support these extensions.
Sources
- 1.Park, T., & Casella, G. (2008). The Bayesian Lasso. Journal of the American Statistical Association, 103(482), 681–686.
- 2.Tibshirani, R. (1996). Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society: Series B, 58(1), 267–288.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Bayesian LASSO Regression. ScholarGate. https://scholargate.app/statistics/bayesian-lasso-regression