Effect Size
The magnitude of an effect, independent of sample size
Effect size is a statistical measure that quantifies the magnitude of a phenomenon independently of sample size. Because p-values conflate effect magnitude with sample size, they are insufficient for judging practical significance. Indices such as Cohen's d, Pearson r, odds ratio, and eta-squared provide researchers with a standardized metric for assessing both practical importance and the essential input for meta-analysis. Conventional small/medium/large benchmarks are rough heuristics, not universal thresholds.
Core Idea and Definition
Effect size expresses numerically the true magnitude of a relationship or difference. The most widely used formula for comparing two group means is Cohen's d: d = (M₁ − M₂) / SD_pooled, where SD_pooled is the combined standard deviation of both groups. For the relationship between two variables, Pearson r or r² (proportion of variance explained) is preferred. For categorical data, the odds ratio or risk ratio is appropriate. All these measures provide a common unit that makes it possible to compare findings across different studies and different measurement instruments.
Distinction from p-Values and Key Differences
Statistical significance (the p-value) indicates how probable the observed data would be under the null hypothesis; however, this value depends on both the effect magnitude and the sample size. In very large samples, even a trivially small and practically meaningless difference can yield p < 0.05; conversely, a real and important effect may not reach statistical significance in a small sample. Effect size separates these two factors by reporting only the quantitative magnitude of the effect, thereby enabling an assessment of practical importance independent of sample size.
Common Misconceptions and Misuses
The most common misconception is that Cohen's benchmarks — d = 0.2 (small), 0.5 (medium), 0.8 (large) — represent universal truths. Cohen proposed these values as rough guidelines for a specific field and era, and in the same work emphasized that they should not be applied blindly. Interpreting effect sizes requires context, domain norms, and practical consequences. Furthermore, effect size alone does not establish causality; a large d obtained from a poorly designed study may simply be the product of methodological error rather than a genuine phenomenon.
Importance in Research Practice
Reporting effect sizes is now required by many journals and institutions, including APA publication guidelines. In meta-analysis, effect sizes are the essential input for meaningfully synthesizing findings across different studies. Moreover, power analysis requires an expected effect size estimate to calculate the necessary sample size. Effect size therefore plays a central role in all three stages of research — planning, reporting, and synthesis — and is an indispensable tool for the cumulative advancement of science.
Key thinkers
- Jacob Cohen (1923–1998)American statistician known for establishing the foundational framework of effect size and statistical power analysis.
Sources
- Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum Associates. ISBN: 978-0-8058-0283-2