Confidence Sets
A confidence set is a data-dependent region that contains the unknown parameter with a guaranteed long-run frequency, providing interval estimates rather than single points.
Definition
A confidence set with level one minus alpha is a random subset of the parameter space, computed from the data, whose probability of containing the true parameter is at least one minus alpha for every value of the parameter.
Scope
This area covers coverage probability and the confidence level, the construction of confidence intervals from pivotal quantities, the duality between confidence sets and hypothesis tests that builds a set by inverting a family of tests, one- and two-sided intervals, the length and expected length of intervals, uniformly most accurate and unbiased confidence sets, and large-sample confidence intervals based on asymptotic normality.
Sub-topics
Core questions
- What does the confidence level mean, and what does it not say about a single computed interval?
- How are confidence intervals constructed from pivotal quantities?
- How does inverting a family of hypothesis tests produce a confidence set?
- What makes one confidence set better than another of the same level?
Key theories
- Pivotal construction
- A pivot is a function of the data and parameter whose distribution is known and free of the parameter; inverting probability statements about the pivot yields confidence intervals with exact coverage.
- Duality of tests and confidence sets
- The set of parameter values not rejected by a level-alpha test is a confidence set of level one minus alpha, and conversely, so optimality of tests transfers to optimality of confidence sets.
Clinical relevance
Confidence intervals are the standard way to report uncertainty in clinical trials, surveys, and measurement science, conveying not just a point estimate but a plausible range, and regulatory and reporting guidelines increasingly require them alongside or instead of p-values.
History
Neyman introduced the theory of confidence intervals in 1937, framing interval estimation as a frequentist coverage guarantee and establishing the duality with hypothesis testing that organizes the subject today.
Debates
- Interpretation of a single confidence interval
- The confidence level is a property of the procedure across repeated samples, not the probability that a particular computed interval contains the parameter; this distinction from Bayesian credible intervals is a recurring source of misinterpretation.
Key figures
- Jerzy Neyman
- Erich L. Lehmann
- George Casella
- Roger L. Berger
Related topics
Seminal works
- casella2002
Frequently asked questions
- Does a 95 percent confidence interval contain the parameter with probability 0.95?
- Not for a single computed interval. The 95 percent refers to the procedure: across many repetitions, about 95 percent of the intervals it produces would contain the true parameter.
- How does a confidence interval differ from a Bayesian credible interval?
- A confidence interval guarantees a coverage frequency over repeated sampling, while a credible interval is a posterior-probability statement about the parameter given the data and a prior; they answer different questions and need not coincide.