Publication Bias

Publication bias is the tendency for studies with statistically significant or positive results to be published, and to be published more prominently and quickly, than studies with null or negative findings. Because systematic reviews and meta-analyses depend on the published record, this selective availability can distort the synthesised estimate toward a stronger effect than the truth.

Definition

Publication bias is the systematic difference between the results of studies that are published and accessible and the results of all studies that have been conducted, arising because the likelihood and prominence of publication depend on the nature and direction of a study's findings.

Scope

This entry covers publication bias and the broader family of reporting biases that affect evidence synthesis: their causes, their effect on pooled estimates, and the graphical and statistical methods used to detect and probe them, including the funnel plot, Egger's regression test, the Begg rank-correlation test, and trim-and-fill adjustment. It treats these as methodological topics, not clinical guidance.

Core questions

• Are the studies available for synthesis a biased sample of all studies conducted?
• How can the presence of such bias be detected from the assembled studies?
• How much might unpublished or selectively reported results change the conclusion?

Key concepts

• Reporting and dissemination bias
• File-drawer problem
• Small-study effects
• Funnel plot and its asymmetry
• Egger's regression test
• Begg's rank-correlation test
• Trim-and-fill adjustment
• Study and protocol registration

Mechanisms

Studies that find significant or favourable results are more likely to be submitted, accepted, and cited than those that do not, so the accessible literature over-represents positive findings. Related reporting biases (selective outcome reporting, time-lag and language bias) work in the same direction. Because a meta-analysis pools whatever it can retrieve, this selection can inflate the summary effect. The funnel plot displays each study's effect against its precision; in the absence of bias the points scatter symmetrically, while a gap among small, less precise studies suggests missing negative results, a pattern often called small-study effects. Egger's regression test and Begg's rank-correlation test quantify funnel-plot asymmetry, and the trim-and-fill method imputes the studies that asymmetry implies are missing and recomputes an adjusted estimate. None of these tools can distinguish publication bias from genuine heterogeneity with certainty, so prospective registration of studies and protocols is the more fundamental safeguard.

Clinical relevance

Publication bias can make an intervention look more effective or safer than the full evidence would show, which matters when reviews inform guidelines and policy. Appraising whether a meta-analysis examined this risk is part of judging its reliability. This entry explains how the bias arises and is investigated; it is reference material for evidence appraisal, not advice for an individual patient.

Epidemiology

Empirical studies tracking cohorts of registered trials and grant-funded studies have repeatedly shown that statistically significant results are published more often and sooner. Funnel-plot examination and asymmetry tests are routinely reported in meta-analyses, and trial registration (for example through prospective registries and journal registration policies) has been adopted in part to counter the problem.

Evidence & guidelines

Recommendations for examining and interpreting funnel-plot asymmetry in meta-analyses of randomised trials were set out by Sterne et al. (2011) and are widely followed; reporting standards such as PRISMA prompt reviewers to assess the risk of bias due to missing results. These are methodological recommendations, not treatment guidance.

History

The under-publication of negative results was described in the 1950s and 1960s and crystallised as the file-drawer problem in the late 1970s. Methods to detect it matured in the 1990s: Begg and Mazumdar (1994) proposed a rank-correlation test of funnel-plot asymmetry, and Egger and colleagues (1997) introduced a simple regression test that became widely used. Duval and Tweedie (2000) added the trim-and-fill adjustment, and Sterne et al. (2011) later consolidated guidance on interpreting funnel plots. Trial registration emerged in parallel as a structural remedy.

Debates

What does funnel-plot asymmetry actually indicate?

Asymmetry can reflect publication bias, but also genuine heterogeneity, differences in study quality, or chance, so asymmetry tests are sensitive to causes other than selective publication and can mislead when studies are few.

How much should trim-and-fill adjustment be trusted?

Trim-and-fill imputes hypothetically missing studies and recomputes the estimate, but it rests on strong assumptions about the shape of the funnel and can over- or under-correct, so it is generally treated as a sensitivity analysis rather than a definitive adjustment.

Key figures

• Matthias Egger
• George Davey Smith
• Colin Begg
• Sue Duval
• Richard Tweedie
• Jonathan Sterne

Seminal works

• egger-1997
• duval-tweedie-2000
• begg-mazumdar-1994
• sterne-2011-funnel

Frequently asked questions

What is a funnel plot and how does it relate to publication bias?

A funnel plot graphs each study's effect estimate against its precision. When no bias is present the studies scatter symmetrically around the pooled effect; a gap among the smaller, less precise studies, especially on the side of unfavourable results, suggests that some negative studies may be missing from the published record.

Can statistical tests prove that publication bias is present?

No. Tests such as Egger's and Begg's detect funnel-plot asymmetry, which can arise from publication bias but also from true heterogeneity, study quality, or chance. They raise or lower suspicion rather than prove the cause, and prospective registration of studies is a stronger safeguard than any after-the-fact test.