Laplace Approximation
Laplace Approximation to the Posterior · Also known as: Laplace's method, saddle-point approximation (Bayesian), second-order Gaussian approximation, LA
The Laplace approximation is a classical analytic technique that replaces an intractable posterior distribution with a multivariate Gaussian centred at the posterior mode, using the curvature of the log-posterior at that mode to set the covariance. Formalised for Bayesian statistics by Tierney and Kadane (1986) in their landmark Journal of the American Statistical Association paper, it provides a fast, deterministic alternative to Markov chain Monte Carlo and forms the mathematical core of Integrated Nested Laplace Approximations (INLA).
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
The Laplace approximation is a good choice when you need fast, deterministic posterior inference without running full MCMC chains. It is most reliable when the posterior is approximately unimodal and not strongly skewed — conditions typically satisfied with large or moderately large samples. It is particularly well suited to generalised linear models (e.g. logistic or Poisson regression) and to any model where the MAP estimate can be obtained by a standard optimiser. It is the default inference engine in INLA for structured additive regression models, making it routine for spatial statistics and epidemiology. It is less reliable for multimodal posteriors, hierarchical models with many latent variables, or posteriors with heavy tails.
Strengths & limitations
- Deterministic and fast: no MCMC chains to run, no convergence monitoring, no thinning.
- Provides an analytic estimate of the log marginal likelihood, useful for model selection and empirical Bayes hyperparameter tuning.
- Scales well to moderate dimensions when the Hessian is sparse, and underpins the highly scalable INLA framework for latent Gaussian models.
- Straightforward to implement: requires only a MAP optimiser and second-order automatic differentiation.
- Approximation quality degrades when the posterior is multimodal, skewed, or has heavy tails — the Gaussian approximation cannot capture these features.
- Hessian computation and inversion cost O(p³) in a dense parameter space, limiting applicability in high-dimensional models without sparsity structure.
- The mode-finding step may converge to a local rather than the global mode in non-convex posteriors.
- Unlike MCMC, the Laplace approximation provides no unbiased estimator of the marginal likelihood; the approximation error is not easily quantified without a ground truth.
Frequently asked
How does the Laplace approximation differ from variational Bayes?
Both are deterministic approximations to the posterior, but they differ in what they optimise. The Laplace approximation fits a Gaussian by matching the location and curvature at the posterior mode. Variational Bayes chooses a distribution from a family by minimising the KL divergence to the true posterior (equivalently, maximising the evidence lower bound). The Laplace approximation is simpler and requires only second-order derivatives, whereas mean-field variational Bayes can be applied to models where computing the Hessian is impractical but tends to underestimate posterior variance.
When is the Laplace approximation exact?
The approximation is exact when the true posterior is itself a multivariate Gaussian — for example, in a linear Gaussian model with a Gaussian prior on coefficients. More generally, by a Bernstein–von Mises theorem argument, the approximation becomes increasingly accurate as sample size grows, because the likelihood dominates the prior and the posterior concentrates into an approximately Gaussian shape around the MLE.
What is INLA and how does it relate to the Laplace approximation?
Integrated Nested Laplace Approximations (INLA), introduced by Rue, Martino, and Chopin (2009), is an inference framework for latent Gaussian models. It uses the Laplace approximation to approximate the conditional posterior of the latent field and the marginal posteriors of the hyperparameters, then integrates over the hyperparameter space numerically. INLA is orders of magnitude faster than MCMC for structured additive regression models and is implemented in the R-INLA package.
Can I use the Laplace approximation for model comparison?
Yes. The log marginal likelihood approximated by the Laplace method can be used to compute Bayes factors or to select among competing models, a strategy MacKay (2003) called the evidence approximation. The approximation is most reliable when the posterior is well-concentrated and approximately Gaussian. For small samples or complex models, cross-validation or exact marginal likelihood estimates from MCMC are preferred.
Sources
- Tierney, L. & Kadane, J. B. (1986). Accurate approximations for posterior moments and marginal densities. Journal of the American Statistical Association, 81(393), 82–86. DOI: 10.1080/01621459.1986.10478240 ↗
- MacKay, D. J. C. (2003). Information Theory, Inference, and Learning Algorithms. Cambridge University Press. ISBN: 978-0521642989
- Rue, H., Martino, S. & Chopin, N. (2009). Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society: Series B, 71(2), 319–392. DOI: 10.1111/j.1467-9868.2008.00700.x ↗
How to cite this page
ScholarGate. (2026, June 3). Laplace Approximation to the Posterior. ScholarGate. https://scholargate.app/en/bayesian/laplace-approximation
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian RegressionBayesian↔ compare
- Expectation PropagationBayesian↔ compare
- MCMCBayesian↔ compare