What is a loss function?

It quantifies the cost of taking a particular action when a particular parameter value is true; common choices are squared error for estimation and zero-one loss for classification, and the risk is its expected value.

Is an admissible rule always a good rule?

Not necessarily. Admissibility only means no other rule dominates it everywhere; some admissible rules are poor overall, and some excellent rules are inadmissible, so admissibility is a minimal rather than a sufficient virtue.

Statistical Decision Theory

Statistical decision theory casts estimation and testing as choices under uncertainty, judged by the expected loss they incur, and asks which decision rules are optimal.

Definition

Statistical decision theory is the framework, due to Wald, in which a statistical procedure is a decision rule mapping data to actions, evaluated by its risk, the expected value of a loss function, and compared with other rules by criteria such as admissibility, minimaxity, and Bayes optimality.

Scope

This area covers loss functions and the risk function as expected loss, the comparison of decision rules, admissibility and inadmissibility, minimax rules that minimize worst-case risk, Bayes rules that minimize average risk under a prior, the relationship between Bayes, minimax, and least-favorable priors, randomized decisions and the geometry of the risk set, and complete-class theorems characterizing the rules worth considering.

Sub-topics

Core questions

How do loss and risk formalize the quality of a statistical procedure?
What does it mean for a decision rule to be admissible or inadmissible?
How are minimax rules related to Bayes rules and least-favorable priors?
Which decision rules form a complete class worth restricting attention to?

Key theories

Risk and admissibility: Each rule has a risk function over the parameter space; a rule is inadmissible if another has no greater risk everywhere and strictly smaller risk somewhere, and admissible otherwise.
Bayes and minimax rules: A Bayes rule minimizes average risk under a prior, a minimax rule minimizes worst-case risk, and under conditions a minimax rule is Bayes against a least-favorable prior, linking the two criteria.
Complete-class theorems: Under convexity and compactness the admissible rules essentially coincide with the Bayes rules and their limits, so attention can be restricted to this complete class without loss.

Clinical relevance

Decision-theoretic risk underlies the comparison of estimators and classifiers by expected loss, the design of cost-sensitive decisions in medical screening and operations, and the principled choice between procedures when no single rule dominates, providing the conceptual backbone for both Bayesian and frequentist methodology.

History

Wald founded statistical decision theory in the 1940s, unifying estimation and testing as decisions under risk and proving early complete-class and minimax results. Blackwell, Stein, and others developed admissibility and the connection to Bayes rules, consolidated in Berger's monograph.

Debates

Minimax versus Bayes criteria: Minimaxity guards against the worst case but can be overly pessimistic, while Bayes optimality depends on a prior that may be hard to justify; decision theory clarifies the trade-off without dictating a single choice.

Key figures

Abraham Wald
James O. Berger
Charles Stein
David Blackwell

Seminal works

berger1985

Frequently asked questions

What is a loss function?: It quantifies the cost of taking a particular action when a particular parameter value is true; common choices are squared error for estimation and zero-one loss for classification, and the risk is its expected value.
Is an admissible rule always a good rule?: Not necessarily. Admissibility only means no other rule dominates it everywhere; some admissible rules are poor overall, and some excellent rules are inadmissible, so admissibility is a minimal rather than a sufficient virtue.