Protocol-based Umbrella Review — Pre-registered Overview of Systematic Reviews
Protocol-based Umbrella Review of Systematic Reviews · Also known as: pre-registered umbrella review, prospero-registered umbrella review, registered overview of reviews, protocol-driven umbrella review
A protocol-based umbrella review is an umbrella review — a synthesis of existing systematic reviews and meta-analyses on a common topic — conducted under a publicly pre-registered protocol, typically in PROSPERO or a similar registry. Pre-registering the protocol before data collection begins commits the research team to prospectively defined eligibility criteria, search strategy, appraisal tools, and synthesis methods, sharply reducing the risk of outcome reporting bias and post-hoc analytical flexibility.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use a protocol-based umbrella review when the literature contains multiple systematic reviews on the same broad question and a high-level, trustworthy synthesis is needed to inform policy, clinical guidelines, or research priority-setting. Pre-registration is especially important when the review is intended to influence practice or when the field has a history of selective reporting. Do NOT use this design when primary studies (rather than existing systematic reviews) are the appropriate evidence unit — in that case a standard systematic review or meta-analysis is more suitable. Also avoid it if the existing systematic reviews are too heterogeneous in population or outcome definitions to compare meaningfully, or if the evidence base is so new that few high-quality systematic reviews exist.
Strengths & limitations
- Pre-registered protocol prevents post-hoc manipulation of eligibility criteria and analytical choices, substantially reducing bias.
- Synthesizes the highest tier of evidence — systematic reviews and meta-analyses — providing a broad, efficient overview across a whole topic area.
- AMSTAR-2 quality appraisal allows transparent weighting of evidence by the methodological rigor of each included review.
- Overlap analysis (CCA) reveals when multiple reviews are drawing on the same primary studies, preventing overconfidence from the illusion of independent evidence.
- The registered protocol creates a clear audit trail for peer reviewers, journal editors, and evidence users.
- Quality of the umbrella review is bounded by the quality of the included systematic reviews; a synthesis of poor-quality reviews remains poor evidence regardless of the umbrella protocol.
- Substantial overlap among primary studies across included reviews can make aggregate effect estimates misleading if not carefully accounted for.
- Pre-registration adds upfront time and constrains adaptability; genuinely unforeseen methodological issues require transparent amendments to the protocol.
- Coverage is limited to questions for which systematic reviews already exist — rapidly emerging topics may not have a sufficient review base.
Frequently asked
What is the difference between a protocol-based umbrella review and a standard umbrella review?
Both designs synthesize systematic reviews on a topic. The protocol-based variant requires that a full, detailed protocol be publicly registered — specifying eligibility criteria, search strategy, appraisal tools, and synthesis methods — before data collection begins. This pre-registration step prevents outcome reporting bias and post-hoc analytical flexibility. A standard umbrella review may follow rigorous methods without public registration, but it lacks the transparency and accountability that registration provides.
Where should I register an umbrella review protocol?
PROSPERO (International Prospective Register of Systematic Reviews, hosted by the University of York) is the most widely used registry for systematic reviews and umbrella reviews in health and social care. The Open Science Framework (OSF) is an acceptable alternative for other disciplines. The registration number must be reported in the published review.
How do I assess the quality of included systematic reviews?
AMSTAR-2 (A Measurement Tool to Assess systematic Reviews, version 2) is the most widely recommended and validated tool for appraising the methodological quality of systematic reviews included in an umbrella review. It covers 16 domains including protocol registration, search comprehensiveness, risk of bias assessment in primary studies, and statistical synthesis methods. Each included review receives an overall confidence rating.
What is corrected covered area (CCA) and why does it matter?
CCA is a metric that quantifies the degree of overlap in primary studies across the systematic reviews included in an umbrella review. When multiple reviews synthesize the same primary studies, the apparent volume of evidence is inflated. A high CCA (above 15% is considered very high) signals that findings across reviews are not truly independent and that aggregate conclusions must be interpreted with caution.
Can I amend a registered protocol after registration?
Yes, but all amendments must be documented, dated, and transparently reported in the final publication. Acceptable reasons include unforeseen issues with data availability or a realisation that the original eligibility criteria were too narrow or too broad. Amendments made to achieve a desired result after seeing preliminary findings undermine the purpose of registration and should be avoided.
Sources
- Aromataris, E., Fernandez, R., Godfrey, C. M., Holly, C., Khalil, H., & Tungpunkom, P. (2015). Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review. JBI Evidence Implementation, 13(3), 132-140. DOI: 10.1097/XEB.0000000000000055 ↗
- Ioannidis, J. P. A. (2009). Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ, 181(8), 488-493. DOI: 10.1503/cmaj.081086 ↗
How to cite this page
ScholarGate. (2026, June 3). Protocol-based Umbrella Review of Systematic Reviews. ScholarGate. https://scholargate.app/en/scientometrics/protocol-based-umbrella-review
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Network Meta-AnalysisEvidence Synthesis↔ compare
- PRISMA-compliant Umbrella ReviewScientometrics↔ compare
- Scoping ReviewScientometrics↔ compare
- Systematic Literature ReviewScientometrics↔ compare
- Umbrella ReviewEvidence Synthesis↔ compare