Publication Bias
Publication Bias and Selective Outcome Reporting in Research Literature · Also known as: file drawer problem, selective reporting, outcome reporting bias, funnel plot asymmetry
Publication bias occurs when the results of a study influence whether the study is published. Typically, studies with statistically significant or positive results are more likely to be published than studies with non-significant or negative results, even if both are scientifically valid. This bias distorts the published literature, making treatments appear more effective than they actually are. Rosenthal (1979) termed this the 'file drawer problem': research with null results sits in file drawers, unpublished, creating a biased sample of published evidence. Funnel plots and statistical tests (e.g., Egger test) can detect asymmetry suggesting publication bias; meta-analyses must account for this bias.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Every meta-analysis should assess publication bias by: (1) Searching comprehensively for unpublished studies. (2) Constructing a funnel plot. (3) Interpreting asymmetry cautiously. Particularly important when: (1) Many small studies exist (small studies are susceptible to publication bias). (2) Effect sizes are heterogeneous. (3) The research question is of high interest (both academic and commercial), increasing motivation to publish positive results selectively. Less critical when: (1) All studies are large, well-powered trials (publication bias is less likely). (2) Null results are common in the literature, suggesting selective publication is not occurring.
Strengths & limitations
- Raises awareness of a pervasive bias affecting published literature, preventing overconfidence in pooled effect estimates.
- Funnel plots provide a simple visual tool for assessing potential bias in meta-analyses.
- Systematic approaches to finding unpublished studies (gray literature search, trial registries) improve the completeness of meta-analyses.
- Trial registration (ClinicalTrials.gov) enables detection of selective outcome reporting by comparing registered protocols to published results.
- Encourages researchers to pre-register hypotheses and outcomes a priori, reducing the motivation for selective reporting.
- Funnel plot asymmetry can result from causes other than publication bias (heterogeneity, outcome reporting bias, study quality), making interpretation ambiguous.
- Comprehensive gray literature searches are time-consuming and may miss unpublished studies nonetheless.
- Trial registries (e.g., ClinicalTrials.gov) are increasingly complete for recent trials but sparse for older studies; retrospective detection of selective reporting is difficult.
- Even with comprehensive searches, non-response bias remains: researchers may fail to find or report unpublished studies, and may not respond to author inquiries.
- Publication bias is difficult to quantify precisely; methods like fail-safe N rely on assumptions about the distribution of unpublished effects.
Frequently asked
What is the difference between publication bias and outcome reporting bias?
Publication bias: an entire study is not published because results are non-significant. The study exists but is not in the literature. Outcome reporting bias: a study IS published, but only certain outcomes (usually positive ones) are reported while other outcomes (often null results) are omitted. Both distort the literature, but outcome reporting bias is harder to detect because the study appears in print. Always compare published papers to registered protocols to detect selective outcome reporting.
How do I search for unpublished studies?
Search: (1) Trial registries: ClinicalTrials.gov (US), ISRCTN (international), EU Clinical Trials Register. (2) Gray literature databases: ProQuest Dissertations & Theses, EThOS (British Library), OpenGrey (European). (3) Preprints: bioRxiv (biology), medRxiv (medicine), arXiv (physics/math). (4) Author contact: email corresponding authors asking about unpublished or in-progress studies. (5) Funding agencies: NIH, NSF, EU Horizon program sometimes post reports of funded research. (6) Conference proceedings. Document all sources; a thorough search is part of meta-analysis quality.
What does funnel plot asymmetry mean?
A funnel plot plots effect size vs. precision (sample size). Absent bias, small studies show wide scatter (low precision) and large studies cluster around the true effect (high precision), forming a symmetric funnel. Asymmetry (e.g., missing small studies with negative results on the left side) suggests publication bias or other issues. However, asymmetry can also reflect true heterogeneity or differences in study quality. Interpret in context; asymmetry is suggestive, not definitive proof of bias.
Can trial registration prevent publication bias?
Partially. Trial registration (e.g., on ClinicalTrials.gov) requires researchers to pre-specify primary and secondary outcomes before data analysis, making selective outcome reporting detectable by comparing registration to publication. However, registration does not prevent the file drawer problem (whole studies not being published). It does enable detection of selective reporting, and regulatory requirements (FDA, EMA) increasingly mandate registration and disclosure.
If I detect funnel plot asymmetry in my meta-analysis, what should I do?
Do not panic—asymmetry does not definitively prove bias. (1) Investigate causes: conduct subgroup analyses by study size, quality, or country to see if asymmetry is explained by heterogeneity. (2) Perform sensitivity analyses: recalculate the pooled effect excluding small studies; if it changes substantially, publication bias may be important. (3) Conduct fail-safe N analysis. (4) Report honestly in your paper: discuss asymmetry, potential causes, and implications. (5) Consider whether additional gray literature searches might find missing studies.
Sources
- Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. DOI: 10.1037/0033-2909.86.3.638 ↗
- Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634. DOI: 10.1136/bmj.315.7109.629 ↗
- Chan, A. W., Hrobjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA, 291(20), 2457–2465. DOI: 10.1001/jama.291.20.2457 ↗
How to cite this page
ScholarGate. (2026, June 3). Publication Bias and Selective Outcome Reporting in Research Literature. ScholarGate. https://scholargate.app/en/research-statistics/publication-bias
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Effect SizeResearch Statistics↔ compare
- Multiple Comparisons ProblemResearch Statistics↔ compare
- Null Hypothesis TestingResearch Statistics↔ compare
- P-Value and Statistical SignificanceResearch Statistics↔ compare