Fisher Exact Randomization Inference
Also known as: fisher randomization test, permutation inference, exact randomization test, randomizasyon çıkarımı (fisher exact randomization)
Randomization inference, introduced by Ronald A. Fisher in The Design of Experiments (1935), computes an exact p-value by evaluating a test statistic across all possible treatment assignments under Fisher's sharp null hypothesis. It is regarded as the gold standard for analysing designed experiments because its validity rests on the known assignment mechanism rather than on distributional assumptions.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use randomization inference when you have a randomized experiment with a known assignment mechanism and want an exact, assumption-light p-value for a treatment effect. It suits comparison and relationship questions with continuous, binary, or categorical outcomes, and needs at least about 10 units so that the permutation set is rich enough. It requires SUTVA (one unit's outcome is unaffected by others' assignments) and that treatment was assigned with known probability. With very small samples the number of possible permutations becomes too limited for a meaningful exact test.
Strengths & limitations
- Yields an exact p-value that follows directly from the known randomization, with no reliance on normality or large-sample approximations.
- Considered the gold standard for analysing designed experiments.
- Works flexibly with continuous, binary, or categorical outcomes and with a freely chosen test statistic.
- Requires SUTVA to hold across units and a known assignment probability; violations undermine the inference.
- With fewer than about 10 units the permutation count is too small for a meaningful exact test.
- Below roughly 5 observations the Fisher exact randomization test is essentially meaningless.
Frequently asked
Why is the p-value called 'exact'?
Because it is computed directly from the finite set of treatment assignments permitted by the randomization, rather than from an approximating distribution. When all permutations are enumerated the p-value is exact; for large designs the permutations are sampled, giving a close approximation.
What is Fisher's sharp null hypothesis?
It states that the treatment has no effect for any single unit — each unit would show the very same outcome whether assigned to treatment or control. This lets you impute every unit's outcome under every reassignment and build the exact reference distribution.
What does SUTVA require?
The Stable Unit Treatment Value Assumption requires that one unit's outcome depends only on its own treatment and not on the assignments of other units, and that there is a single, well-defined version of each treatment. If units interfere with each other, the permutation logic breaks down.
What if my sample is too small?
With fewer than about 10 units the permutation set is too limited for a meaningful exact test, and below 5 it is essentially meaningless. In those cases bootstrap inference or a jackknife is the recommended fallback.
Sources
- Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd. link ↗
- Imbens, G. W. & Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences. Cambridge University Press. ISBN: 978-0521885881
How to cite this page
ScholarGate. (2026, June 1). Fisher Exact Randomization Inference. ScholarGate. https://scholargate.app/en/statistics/randomization-inference
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bootstrap InferenceStatistics↔ compare
- JackknifeStatistics↔ compare
- Nonparametric Quantile RegressionStatistics↔ compare
- OLS RegressionEconometrics↔ compare
- Permutation TestStatistics↔ compare