Point Estimation

Point estimation studies how to summarize data by a single best guess of an unknown parameter, and how to judge whether one estimator is better than another.

Definition

Point estimation is the branch of statistical inference concerned with using observed data to produce a single value, called a point estimate, as the best available approximation to an unknown population parameter.

Scope

This area covers the reduction of data through sufficient and complete statistics, the construction of estimators by maximum likelihood and the method of moments, the evaluation of estimators through bias, variance, and mean squared error, the Cramer-Rao information bound and the notion of efficiency, the Rao-Blackwell and Lehmann-Scheffe routes to minimum-variance unbiased estimators, and Bayes and shrinkage estimators that trade bias for reduced risk.

Sub-topics

• bayes and shrinkage estimation
• maximum likelihood estimation
• sufficiency and completeness
• unbiased estimation and cramer rao

Core questions

• How can a sample be reduced to a sufficient statistic without losing information about the parameter?
• What makes one estimator better than another, and how do bias and variance combine in mean squared error?
• How low can the variance of an unbiased estimator be, and when is that bound attained?
• When does shrinking an estimator toward a prior or a fixed point reduce its overall risk?

Key theories

Sufficiency and the factorization theorem

A sufficient statistic captures all the sample information about a parameter; the factorization theorem identifies sufficiency from the way the likelihood depends on the data and the parameter, and completeness yields uniqueness of unbiased estimators.

Maximum likelihood estimation

Estimating the parameter that makes the observed data most probable; under regularity conditions the maximum likelihood estimator is consistent, asymptotically normal, and asymptotically efficient.

Cramer-Rao bound and efficiency

The variance of any unbiased estimator is bounded below by the reciprocal of the Fisher information; an estimator attaining this bound is efficient, and the Rao-Blackwell and Lehmann-Scheffe theorems construct minimum-variance unbiased estimators.

Clinical relevance

Point estimators are the workhorse of applied quantitative science: maximum likelihood underlies the fitting of statistical and machine-learning models, shrinkage estimators improve prediction in high-dimensional problems, and the Fisher information governs how precisely experiments can resolve a parameter, informing sample-size and experimental-design decisions.

History

Fisher introduced likelihood, sufficiency, information, and efficiency in the 1920s, founding the modern theory of estimation. Rao and Cramer established the variance bound around 1945, Rao and Blackwell and later Lehmann and Scheffe completed the unbiased-estimation theory, and Stein's 1956 discovery of inadmissibility in three or more dimensions opened the study of shrinkage.

Key figures

• Ronald A. Fisher
• Calyampudi Radhakrishna Rao
• Erich L. Lehmann
• Charles Stein

Seminal works

• lehmannCasella1998

Frequently asked questions

What is the difference between an estimator and an estimate?

An estimator is a rule or function of the data, viewed as a random variable before the data are seen; an estimate is the particular numerical value the estimator takes once the data are observed.

Is an unbiased estimator always the best choice?

Not necessarily. A biased estimator can have smaller mean squared error than the best unbiased one, which is why shrinkage and Bayes estimators are often preferred when overall accuracy matters more than zero bias.