Likelihood-Ratio Tests

The likelihood-ratio test compares how well the data fit under a restricted null model versus the full model, and its large-sample distribution makes it a universal testing tool.

Definition

The generalized likelihood-ratio test rejects the null hypothesis when the ratio of the maximized likelihood under the null to the maximized likelihood over the full parameter space is small, equivalently when minus twice its logarithm is large.

Scope

This topic covers the generalized likelihood-ratio statistic for composite hypotheses, Wilks's theorem that minus twice the log-likelihood-ratio is asymptotically chi-squared with degrees of freedom equal to the number of constraints, the asymptotically equivalent Wald and score (Rao) tests, the relations and differences among the three, and the regularity conditions and exceptions such as parameters on a boundary.

Core questions

• How is the generalized likelihood-ratio statistic constructed for composite hypotheses?
• Why is minus twice the log-likelihood-ratio asymptotically chi-squared, as in Wilks's theorem?
• How do the Wald and score tests relate to the likelihood-ratio test?
• When does the standard chi-squared approximation fail?

Key theories

Wilks's theorem

Under the null and regularity conditions, minus twice the logarithm of the likelihood ratio converges to a chi-squared distribution whose degrees of freedom equal the number of restrictions imposed by the null.

Wald, score, and likelihood-ratio trinity

The Wald test uses the distance of the estimate from the null, the score test uses the gradient of the log-likelihood at the null, and the likelihood-ratio test uses the difference in maxima; all three share the same asymptotic chi-squared distribution.

Clinical relevance

Likelihood-ratio, Wald, and score tests are the standard significance tests reported by software for regression coefficients, nested-model comparisons, and goodness-of-fit, making them the everyday inferential tools across epidemiology, econometrics, and the experimental sciences.

History

Wilks established the asymptotic chi-squared distribution of the likelihood-ratio statistic in 1938. Wald introduced his test in 1943 and Rao the score test in 1948, and the asymptotic equivalence of the three was clarified through the mid-twentieth century.

Key figures

• Samuel S. Wilks
• Abraham Wald
• Calyampudi Radhakrishna Rao
• Aad van der Vaart

Seminal works

• casella2002

Frequently asked questions

When do the Wald, score, and likelihood-ratio tests disagree?

They are asymptotically equivalent but can differ in finite samples; the likelihood-ratio and score tests are generally more reliable than the Wald test when the likelihood is far from quadratic or estimates are near a boundary.

What are the degrees of freedom in Wilks's theorem?

They equal the number of independent constraints the null hypothesis places on the parameters, that is, the difference in dimension between the full and the restricted models.