Fisher Exact Randomization Inference
Also known as: fisher randomization test, permutation inference, exact randomization test, randomizasyon çıkarımı (fisher exact randomization)
Randomization inference, introduced by Ronald A. Fisher in The Design of Experiments (1935), computes an exact p-value by evaluating a test statistic across all possible treatment assignments under Fisher's sharp null hypothesis. It is regarded as the gold standard for analysing designed experiments because its validity rests on the known assignment mechanism rather than on distributional assumptions.
Key highlights
- Yields an exact p-value that follows directly from the known randomization, with no reliance on normality or large-sample approximations.
- Considered the gold standard for analysing designed experiments.
- Works flexibly with continuous, binary, or categorical outcomes and with a freely chosen test statistic.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use randomization inference when you have a randomized experiment with a known assignment mechanism and want an exact, assumption-light p-value for a treatment effect. It suits comparison and relationship questions with continuous, binary, or categorical outcomes, and needs at least about 10 units so that the permutation set is rich enough. It requires SUTVA (one unit's outcome is unaffected by others' assignments) and that treatment was assigned with known probability. With very small samples the number of possible permutations becomes too limited for a meaningful exact test.
Strengths & limitations
- Yields an exact p-value that follows directly from the known randomization, with no reliance on normality or large-sample approximations.
- Considered the gold standard for analysing designed experiments.
- Works flexibly with continuous, binary, or categorical outcomes and with a freely chosen test statistic.
- Requires SUTVA to hold across units and a known assignment probability; violations undermine the inference.
- With fewer than about 10 units the permutation count is too small for a meaningful exact test.
- Below roughly 5 observations the Fisher exact randomization test is essentially meaningless.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
Why is the p-value called 'exact'?
Because it is computed directly from the finite set of treatment assignments permitted by the randomization, rather than from an approximating distribution. When all permutations are enumerated the p-value is exact; for large designs the permutations are sampled, giving a close approximation.
What is Fisher's sharp null hypothesis?
It states that the treatment has no effect for any single unit — each unit would show the very same outcome whether assigned to treatment or control. This lets you impute every unit's outcome under every reassignment and build the exact reference distribution.
What does SUTVA require?
The Stable Unit Treatment Value Assumption requires that one unit's outcome depends only on its own treatment and not on the assignments of other units, and that there is a single, well-defined version of each treatment. If units interfere with each other, the permutation logic breaks down.
What if my sample is too small?
With fewer than about 10 units the permutation set is too limited for a meaningful exact test, and below 5 it is essentially meaningless. In those cases bootstrap inference or a jackknife is the recommended fallback.
Sources
- 1.Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd.
- 2.Imbens, G. W. & Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences. Cambridge University Press.ISBN 978-0521885881
You have read it. What now?
Cite this page
ScholarGate. (2026, June 1). Randomization Inference. ScholarGate. https://scholargate.app/statistics/randomization-inference