Sensitivity Analysis

Sensitivity analysis in evidence synthesis is the practice of repeating the meta-analysis under different reasonable assumptions or with certain studies removed, to see whether the main conclusion holds. It probes how dependent a pooled result is on particular choices, particular studies, or particular data, and so gauges the robustness of the synthesis.

Definition

Sensitivity analysis is a set of procedures that re-estimate a pooled effect under alternative analytic assumptions or with selected studies or data excluded, in order to assess whether and how the conclusion of a meta-analysis changes.

Scope

This entry covers sensitivity analysis as applied within meta-analysis and systematic reviews: leave-one-out and influence analyses, comparing fixed-effect with random-effects results, restricting to low-risk-of-bias studies, and testing the effect of missing or imputed data. It is a methodological reference description and not clinical guidance. A separate sensitivity-analysis node covers the concept as used in causal inference and modelling.

Core questions

• Does the pooled conclusion depend on any single influential study?
• Does it change when fixed-effect and random-effects models are compared?
• How robust is it to including or excluding studies at high risk of bias?
• How sensitive is it to assumptions about missing data or alternative effect measures?

Key concepts

• Leave-one-out (influence) analysis
• Fixed-effect versus random-effects comparison
• Risk-of-bias restriction
• Missing-data and imputation assumptions
• Robustness of the pooled estimate

Mechanisms

The synthesis is recomputed under a deliberately varied assumption while everything else is held fixed, and the analyst observes whether the pooled estimate, its interval, or the qualitative conclusion shifts. Common variants include removing one study at a time to detect an outlier or influential trial, switching between fixed-effect and random-effects models to see how much heterogeneity assumptions matter (a comparison Riley and colleagues note can change the apparent precision substantially), restricting the synthesis to studies judged at low risk of bias, and re-running the analysis under alternative handling of missing data or different effect metrics. A result that stays essentially unchanged across these variations is considered robust; one that moves materially signals that the conclusion is contingent and must be reported with that caveat. Related diagnostics such as examining funnel-plot asymmetry, as set out by Sterne and colleagues, complement sensitivity analysis by probing whether small-study effects or reporting bias threaten the pooled estimate.

Clinical relevance

Whether a guideline or health technology assessment should act on a pooled estimate depends partly on how robust that estimate is, so sensitivity analyses help readers judge how much confidence a synthesised result warrants. This entry describes the method and is not a basis for individual clinical decisions.

Evidence & guidelines

Sensitivity analysis is an expected component of systematic-review conduct under the Cochrane Handbook (Higgins & Green, 2008) and is reflected in PRISMA reporting items (Moher et al., 2009); guidance on the related assessment of funnel-plot asymmetry is given by Sterne and colleagues (2011).

History

As clinical meta-analysis matured through the 1990s and 2000s, reviewers increasingly recognised that a single pooled number could hide fragility, and pre-specified sensitivity analyses became part of standardised review conduct codified in the Cochrane Handbook and PRISMA. Parallel work on funnel-plot asymmetry (Sterne et al., 2011) gave structured ways to test one important threat, small-study effects, within the same robustness-checking spirit.

Debates

How should sensitivity analyses be planned and reported?

There is broad agreement that sensitivity analyses should be pre-specified and clearly distinguished from post-hoc exploration to avoid selectively reporting the version that supports a desired conclusion, a standard reflected in current reporting guidance.

Key figures

• Julian Higgins
• Jonathan Sterne
• Richard Riley
• Jonathan Deeks

Seminal works

• higgins-handbook-2008
• sterne-2011

Frequently asked questions

What is a leave-one-out sensitivity analysis?

It repeats the meta-analysis several times, each time omitting one study, to see whether any single study is driving the pooled result; if the estimate is stable across all of these, no single study is unduly influential.

How is sensitivity analysis different from subgroup analysis?

Sensitivity analysis tests whether a conclusion is robust to analytic choices or to excluding studies, whereas subgroup analysis explores whether the effect itself differs between defined groups of studies or participants.