Questionable Research Practices

p-hacking, HARKing, selective reporting

Questionable research practices (QRPs) fall short of outright fraud yet systematically distort the scientific literature. Practices such as p-hacking, HARKing, and selective reporting inflate false-positive rates and are among the primary drivers of the replication crisis. Awareness of these practices and adoption of transparency measures — pre-registration and full outcome reporting — are essential for maintaining scientific credibility.

Definition of the Concept

Questionable research practices refer to patterns of behavior in which researchers do not fabricate data outright but make biased decisions during analysis or reporting. Although not classified as explicit ethical violations, these practices systematically distort the direction and magnitude of scientific findings. The term gained widespread use following John et al.'s 2012 study and has since become a central focus of meta-scientific research in psychology, medicine, and the social sciences.

Main Types and How They Work

The most common QRP types are: (1) p-hacking — trying different subgroups, covariates, or analysis methods until p < 0.05 is achieved; (2) HARKing (Hypothesizing After Results are Known) — presenting post-hoc hypotheses as if they were formulated a priori; (3) selective reporting — publishing only statistically significant outcomes while filing away null results; (4) optional stopping — terminating data collection as soon as p < 0.05 is reached. Each of these inflates the false-positive rate in the published literature.

A Concrete Example

A researcher measures the effect of an intervention on anxiety using three different scales. After analysis, only one scale yields a significant result; the other two are omitted from the manuscript. Additionally, although 100 participants were originally planned, data collection is stopped after participant 65 because p < 0.05 is reached at that point. These two decisions — selective reporting and optional stopping — together substantially increase the probability that the published finding is a false positive.

Remedies and Common Misconceptions

The most effective remedy against QRPs is pre-registration: publicly logging hypotheses, sample sizes, and analysis plans on a platform such as OSF or AsPredicted before data collection begins. Full outcome reporting and data sharing further enhance transparency. A common misconception is that these practices apply only to dishonest researchers; in reality, many QRPs occur unconsciously. Another misconception is that small sample size is the sole problem; the deeper issue is the flexible, undisclosed execution of analyses.

Key terms

p-hacking: Trying multiple analyses until p < 0.05 is reached to achieve statistical significance.
HARKing: Presenting post-hoc hypotheses as if they were formulated before seeing the results.
Selective Reporting: Publishing only significant findings while withholding null or nonsignificant results.
Pre-registration: Publicly logging hypotheses and analysis plans before data collection begins.
Replication Crisis: Scientific credibility problem arising when published findings cannot be reproduced in replication studies.