Jackknife Resampling
The jackknife estimates the bias and variance of a statistic by systematically recomputing it on the data sets obtained by leaving out one observation at a time.
Definition
The jackknife is a resampling method that computes a statistic on each subsample formed by deleting one observation, then combines these leave-one-out values into estimates of the statistic's bias and variance.
Scope
This topic covers the leave-one-out jackknife, the pseudo-values it generates, jackknife estimates of bias and standard error, the delete-d generalization, and the relationship between the jackknife and the bootstrap as linear-approximation and full resampling estimators. Settings where the jackknife is unreliable, such as non-smooth statistics, are noted.
Core questions
- How are leave-one-out recomputations turned into estimates of bias and standard error?
- What are pseudo-values and how do they summarize each observation's influence?
- How does the delete-d jackknife handle statistics for which the simple jackknife fails?
- How is the jackknife related to the bootstrap as a linear approximation?
Key concepts
- Leave-one-out subsamples
- Pseudo-values
- Jackknife bias estimate
- Jackknife variance estimate
- Delete-d jackknife
Key theories
- Leave-one-out estimation
- Recomputing a statistic with each observation removed yields a set of perturbed values whose spread estimates variance and whose mean shift, scaled by sample size, estimates bias.
- Relation to the bootstrap
- The jackknife can be viewed as a linear approximation to the bootstrap, accurate for smooth statistics but failing for non-smooth ones such as the median, which motivated the delete-d generalization.
Clinical relevance
The jackknife gives quick bias and variance estimates that require only as many recomputations as there are observations, and its pseudo-values double as influence diagnostics for detecting observations that disproportionately affect an estimate.
History
Quenouille proposed leave-one-out recomputation for bias reduction around 1949, and Tukey extended it in the 1950s into a general tool for variance estimation, coining the name jackknife; Efron later placed it within the broader resampling framework alongside the bootstrap.
Key figures
- Maurice Quenouille
- John Tukey
- Rupert Miller
- Bradley Efron
Related topics
Seminal works
- efron1979
- miller1974
Frequently asked questions
- How does the jackknife differ from the bootstrap?
- The jackknife uses the fixed set of leave-one-out subsamples, while the bootstrap draws many random samples with replacement. The jackknife is faster and deterministic but is only a linear approximation, and it can fail for non-smooth statistics where the bootstrap still works.
- Why can the simple jackknife fail for the median?
- The median changes in jumps rather than smoothly as single points are removed, so the leave-one-out values do not capture its variability well. Deleting larger groups of observations, the delete-d jackknife, restores a usable estimate.