Robust Effect Size Analysis
Also known as: robust Cohen's d, trimmed-mean effect size, outlier-resistant effect size, robust standardized mean difference
Robust effect size analysis quantifies the magnitude of a difference or association using estimators that are resistant to outliers and violations of normality. Rather than relying on classical statistics such as Cohen's d based on sample means and standard deviations, robust variants use trimmed means and Winsorized standard deviations to produce effect size estimates that accurately reflect the typical effect rather than being inflated by extreme values.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use robust effect size analysis whenever data contain outliers, show heavy-tailed distributions, or fail normality checks, and you need to report the practical magnitude of a group difference or association. It is particularly recommended in psychology, education, clinical research, and any domain where extreme scores are substantively plausible. Do NOT use it when data are well-behaved (normal, no outliers) and sample sizes are adequate — classical Cohen's d is simpler and equally valid in those cases. Also avoid robust effect sizes when the trimming removes substantively meaningful data rather than artifacts, or when audiences or journals require only classical effect size metrics.
Strengths & limitations
- Resistant to outliers and heavy-tailed distributions that severely distort classical Cohen's d.
- Trimmed mean and Winsorized standard deviation are both well-studied estimators with known statistical properties.
- Provides a meaningful, interpretable magnitude estimate even when distributional assumptions are violated.
- Naturally pairs with robust hypothesis tests (e.g., Yuen's t on trimmed means) for consistent reporting.
- Bootstrap confidence intervals require no additional distributional assumptions beyond the observed data.
- Reduces risk of reporting inflated effect sizes driven by a few atypical observations.
- Trimming discards real data; the 20% default may not be appropriate for all datasets or research questions.
- Less familiar to reviewers and readers than Cohen's d, requiring additional explanation in manuscripts.
- Software support is less universal than for classical effect sizes; WRS2 (R) or custom code often required.
- When the distribution is genuinely normal and outlier-free, robust estimators are slightly less efficient than classical ones.
- Interpretation in terms of benchmarks (small/medium/large) may not translate directly from Cohen's original thresholds.
Frequently asked
What trimming proportion should I use?
The default 20% trimming is the most studied and recommended starting point. It performs well across a wide range of distributions. For very small samples, lower proportions (5–10%) may be preferable. Sensitivity analyses with different proportions are advisable if your choice is consequential.
How does robust Cohen's d compare to rank-based effect sizes like the rank-biserial correlation?
Both are resistant to outliers, but they measure different things. Robust d is still a standardized mean difference on the original scale; rank-biserial (or probability of superiority) is a non-parametric effect size measuring how often one group exceeds the other. Use rank-based measures when the original scale has no meaningful interpretation; use robust d when you want to stay on the original measurement scale.
Can I use robust effect sizes in a meta-analysis?
Yes, but with caution. Primary studies must report robust effect sizes consistently (same trimming proportion, same estimator) for pooling to be valid. If primary studies report only classical d, post-hoc recomputation from raw data is needed, which is rarely feasible.
Is there a software package for robust effect sizes?
The WRS2 package in R implements robust effect sizes, including Algina-Keselman-Penfield's estimator, with associated bootstrap confidence intervals. The akp.effect function directly computes the robust standardized mean difference for two independent groups.
Do I still report classical Cohen's d alongside the robust version?
Reporting both can be informative, especially if they diverge substantially — divergence signals that outliers are materially affecting the classical estimate. However, when data are clearly non-normal or contaminated, the robust estimate is the more trustworthy summary and should be foregrounded.
Sources
- Algina, J., Keselman, H. J., & Penfield, R. D. (2005). An alternative to Cohen's standardized mean difference effect size: A robust parameter and confidence interval in the two independent groups case. Psychological Methods, 10(3), 317–328. DOI: 10.1037/1082-989X.10.3.317 ↗
- Wilcox, R. R. (2012). Introduction to Robust Estimation and Hypothesis Testing (3rd ed.). Academic Press. ISBN: 978-0123869838
How to cite this page
ScholarGate. (2026, June 3). Robust Effect Size Analysis. ScholarGate. https://scholargate.app/en/statistics/robust-effect-size-analysis
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Effect size analysisStatistics↔ compare
- Power analysisStatistics↔ compare
- Robust Descriptive StatisticsStatistics↔ compare
- Robust independent samples t-testStatistics↔ compare
- Robust one-way ANOVAStatistics↔ compare