When should I use quadrature instead of Monte Carlo for a statistical integral?

For low-dimensional integrals with smooth integrands, deterministic quadrature converges much faster and gives a deterministic answer. Monte Carlo becomes preferable as the dimension grows, where quadrature grids become impractical.

What is the Laplace approximation good for?

It gives a fast closed-form approximation to integrals dominated by a single sharp peak, such as marginal likelihoods in well-identified models. It is accurate when the integrand is approximately Gaussian near its mode.

Numerical Integration in Statistics

Numerical integration in statistics evaluates the integrals that define marginal likelihoods, posterior expectations and normalizing constants when those integrals have no closed form.

یافتن موضوع با PaperMindبه‌زودیFind papers & topics

Tools & resources

دریافت اسلایدها

Learn & explore

ویدیوبه‌زودی

Definition

Numerical integration in statistics is the use of deterministic quadrature rules and analytic approximations to evaluate the integrals arising in likelihood-based and Bayesian inference, particularly marginal likelihoods and posterior moments.

Scope

This topic covers deterministic quadrature adapted to statistical integrands, including Gauss-Hermite rules for integrating out normal random effects, adaptive quadrature, and the Laplace approximation for integrals dominated by a sharp peak. It complements Monte Carlo integration, which is treated under Monte Carlo methods, by focusing on low-dimensional deterministic schemes.

Core questions

How are random effects integrated out of a likelihood using Gaussian quadrature?
When does adaptive quadrature outperform fixed rules for statistical integrands?
How does the Laplace approximation exploit a sharply peaked integrand?
When are deterministic quadrature methods preferable to Monte Carlo integration?

Key concepts

Gauss-Hermite quadrature
Adaptive quadrature
Laplace approximation
Marginal likelihood
Normalizing constant

Key theories

Gauss-Hermite quadrature for random effects: Integrals against a normal density, such as those marginalizing random effects in mixed models, are evaluated efficiently by Gauss-Hermite rules, with adaptive versions centering the nodes near the integrand's mode.
Laplace approximation: Approximating a sharply peaked integrand by a Gaussian around its mode yields a closed-form estimate of the integral, accurate when the peak dominates, and underlies fast approximate inference for many hierarchical models.

Clinical relevance

Fitting generalized linear mixed models, computing Bayes factors, and obtaining posterior summaries all require evaluating intractable integrals; deterministic quadrature and the Laplace approximation provide fast, accurate alternatives to simulation for low-dimensional integrals.

History

Classical quadrature and Laplace's method of approximating integrals were adapted by statisticians for likelihood and Bayesian computation, with adaptive Gauss-Hermite quadrature and the Laplace approximation becoming standard tools for mixed and hierarchical models.

Key figures

John Monahan
Kenneth Lange
Pierre-Simon Laplace

Seminal works

monahan2011
lange2010

Frequently asked questions

When should I use quadrature instead of Monte Carlo for a statistical integral?: For low-dimensional integrals with smooth integrands, deterministic quadrature converges much faster and gives a deterministic answer. Monte Carlo becomes preferable as the dimension grows, where quadrature grids become impractical.
What is the Laplace approximation good for?: It gives a fast closed-form approximation to integrals dominated by a single sharp peak, such as marginal likelihoods in well-identified models. It is accurate when the integrand is approximately Gaussian near its mode.