Publication Bias

Publication bias is the distortion that arises when whether and how a study is published depends on its results — typically because statistically significant or positive findings are more likely to appear in the literature than null or negative ones. Because evidence synthesis can only combine the studies it can find, publication bias can inflate pooled estimates and is one of the central threats to the validity of a systematic review or meta-analysis.

Definition

Publication bias is the systematic difference between the results of studies that are published and the results of all studies conducted, occurring when the likelihood of publication depends on the direction or significance of a study's findings.

Scope

This topic covers what publication bias is, how it arises, how it is detected (funnel plots and asymmetry tests) and adjusted for (such as trim-and-fill), and how it is reflected in certainty ratings. It also notes related reporting biases. It is a methodological reference, not clinical guidance.

Core questions

• Why are positive or significant results more likely to be published?
• How can missing studies be detected from the studies that are available?
• When does funnel-plot asymmetry indicate publication bias versus other causes?
• How should suspected publication bias affect confidence in a synthesis?

Key concepts

• Outcome-dependent publication
• Small-study effects
• Funnel plot and funnel-plot asymmetry
• Egger's regression test
• Trim-and-fill adjustment
• File-drawer problem
• Selective outcome reporting
• Trial registration as prevention

Mechanisms

Publication bias arises when studies with statistically significant or favourable results are submitted, accepted, and published more readily and quickly than those with null results, so the published record over-represents positive findings. In a meta-analysis this often appears as small-study effects: smaller, less precise studies show systematically larger effects than larger ones. A funnel plot displays effect estimates against their precision; symmetric scatter is expected under no bias, while asymmetry can signal missing small negative studies, formalised by tests such as Egger's regression. Adjustment methods such as trim-and-fill estimate and impute the studies that may be missing. Asymmetry has causes other than publication bias, so detection requires careful interpretation, and prospective trial registration is the principal preventive measure (egger-1997; duval-tweedie-2000; sterne-2011; dickersin-1990).

Clinical relevance

Because guidelines and health technology assessments rest on synthesised evidence, publication bias can propagate into recommendations by overstating benefit or understating harm; appraising whether a synthesis addressed it is part of evidence appraisal. The concept describes a threat to the evidence base and informs how certainty is rated; it is not itself clinical advice.

Evidence & guidelines

Methods and recommendations are well established: Egger's test for funnel-plot asymmetry, the trim-and-fill adjustment, and consensus recommendations on examining and interpreting funnel plots; publication bias is also a domain that lowers certainty in the GRADE framework (egger-1997; duval-tweedie-2000; sterne-2011; guyatt-2008-grade).

History

Concern that the literature over-represents positive findings was articulated as the file-drawer problem in 1979 and documented empirically for medical research by Dickersin in 1990. Detection methods followed: the funnel plot and Egger's regression test in 1997 and the trim-and-fill adjustment in 2000, with consensus recommendations on interpreting funnel-plot asymmetry published in 2011. Trial registration emerged as the leading structural remedy (dickersin-1990; egger-1997; duval-tweedie-2000; sterne-2011).

Debates

Does funnel-plot asymmetry really indicate publication bias?

Asymmetry can also stem from true heterogeneity, study quality, or chance, so asymmetry tests can mislead, especially with few studies; recommendations stress cautious interpretation rather than mechanical adjustment.

Key figures

• Kay Dickersin
• Matthias Egger
• George Davey Smith
• Jonathan Sterne
• Sue Duval

Seminal works

• egger-1997
• duval-tweedie-2000
• dickersin-1990

Frequently asked questions

How is publication bias detected in a meta-analysis?

Commonly through a funnel plot, which plots study effect against precision; asymmetry, tested formally with methods such as Egger's regression, can suggest that small negative studies are missing — though asymmetry has other possible causes.

How can publication bias be prevented?

The leading structural remedy is prospective registration of studies and pre-specification of outcomes, so that the existence and planned analyses of a study are recorded regardless of whether its results are later published.