Sufficiency and Completeness

A sufficient statistic compresses a sample without discarding any information about the parameter; completeness adds the uniqueness that turns such compression into optimal estimation.

Definition

A statistic is sufficient for a parameter if the conditional distribution of the data given the statistic does not depend on the parameter; it is complete if no nontrivial function of it has expectation zero for every parameter value.

Scope

This topic covers the definition of sufficiency, the Fisher-Neyman factorization theorem, minimal sufficient statistics and how to find them, complete and bounded-complete statistics, the role of the exponential family, ancillary statistics, and Basu's theorem on the independence of a complete sufficient statistic from any ancillary statistic.

Core questions

• How does the factorization theorem let one read sufficiency directly off the likelihood?
• What is a minimal sufficient statistic, and how is it constructed?
• Why does completeness guarantee that an unbiased function of the statistic is unique?
• How does Basu's theorem use completeness to prove independence without computation?

Key theories

Factorization theorem

A statistic is sufficient if and only if the joint density factors into a part depending on the data only through that statistic and the parameter, and a part depending only on the data.

Completeness and Basu's theorem

Completeness ensures uniqueness of unbiased estimators based on the statistic; Basu's theorem states that a complete sufficient statistic is independent of every ancillary statistic.

Clinical relevance

Reducing data to a sufficient statistic justifies summarizing large datasets by a few numbers without information loss, which underlies efficient storage, the design of summary reports, and the construction of optimal estimators used throughout applied statistics.

History

Fisher introduced sufficiency in 1922 as the property that a statistic loses no information. Neyman gave the factorization criterion, and Lehmann and Scheffe developed completeness in the 1950s; Basu proved his independence theorem in 1955, tying the concepts together.

Key figures

• Ronald A. Fisher
• Jerzy Neyman
• Debabrata Basu
• Erich L. Lehmann

Seminal works

• lehmannCasella1998

Frequently asked questions

Why is a sufficient statistic useful?

It lets you replace the full dataset with a smaller summary while keeping every bit of information the data carry about the parameter, simplifying inference without any loss.

What is an ancillary statistic?

A statistic whose distribution does not depend on the parameter; by Basu's theorem it is independent of any complete sufficient statistic, which is often used to simplify probability calculations.