Minimax Estimation

A minimax estimator minimizes the largest risk it can incur, offering a guarantee against the worst case when no prior over the parameter is assumed.

Definition

A minimax decision rule is one whose maximum risk over the parameter space is as small as that of any other rule; it minimizes the worst-case expected loss and is typically Bayes against a least-favorable prior.

Scope

This topic covers the minimax criterion of minimizing worst-case risk, least-favorable prior distributions, the characterization of a minimax rule as a Bayes rule with constant risk against a least-favorable prior, the minimax theorem and the game-theoretic value of the statistical game, the use of limiting priors, and minimax rates of convergence that describe the best achievable risk in nonparametric and high-dimensional problems.

Core questions

• What does it mean to minimize worst-case risk, and when is this an appropriate criterion?
• What is a least-favorable prior, and how does it identify a minimax rule?
• Why is a Bayes rule with constant risk automatically minimax?
• What are minimax rates of convergence in nonparametric problems?

Key theories

Minimax rules and least-favorable priors

A rule that is Bayes against a prior and has constant risk is minimax, and that prior is least favorable; this characterization is the main tool for finding minimax estimators.

Minimax rates of convergence

In nonparametric and high-dimensional problems the minimax risk decreases at a rate determined by the smoothness or sparsity of the class, giving a benchmark for the best possible estimation accuracy.

Clinical relevance

Minimax rates set the gold-standard benchmark for nonparametric regression, density estimation, and high-dimensional methods, telling practitioners the best accuracy attainable for a given smoothness or sparsity and whether a proposed estimator is rate-optimal.

History

Wald introduced the minimax criterion and its game-theoretic reading in the 1940s. The theory of least-favorable priors matured at mid-century, and Le Cam, Pinsker, and later authors developed minimax rates for nonparametric problems in the following decades.

Key figures

• Abraham Wald
• Lucien Le Cam
• Charles Stein
• James O. Berger

Seminal works

• berger1985

Frequently asked questions

When is the minimax criterion appropriate?

When robustness against the worst case matters and no reliable prior is available; it can be overly conservative if the worst case is implausible, so it is one criterion among several rather than a universal rule.

What is a least-favorable prior?

It is the prior that makes the estimation problem hardest, maximizing the Bayes risk; a Bayes rule against it with constant risk is minimax, which is why finding it is the key to minimax estimation.