Randomization Test for Single-Case Designs
Also known as: Single-Case Randomization Test, Edgington Randomization Test, Permutation Test for Single-Subject Designs, Single-Case Permutation Inference
The randomization test for single-case experimental designs is a permutation-based procedure that yields a valid statistical p-value for an intervention effect in a single participant, provided that some experimentally controllable feature of the design — typically the moment the intervention begins or the order in which conditions are presented — was randomly determined before data were collected. Eugene Edgington showed in 1980 that this design-embedded randomization is what licenses inference: because the random assignment is the source of the test's probability statements, the procedure draws valid conclusions without assuming that the data are normally distributed or serially independent, two assumptions that single-case time-series data routinely violate. Edgington and Onghena's monograph established the modern framework, in which the observed test statistic is referred to the distribution of statistics generated by every admissible re-assignment of the data. In disability research, where interventions are often delivered to one person at a time and group designs are impractical, the randomization test provides a defensible significance test that complements visual analysis.
Key highlights
- Provides an exact, distribution-free significance test whose validity depends only on the randomization built into the design, not on normality or independence.
- Directly suited to single-case and small-n disability research where group designs are infeasible and data are autocorrelated.
- Flexible in the choice of test statistic, allowing the analyst to target the specific predicted pattern of change.
- Complements visual analysis with a formal probability statement, strengthening the evidentiary value of single-case studies.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use a single-case randomization test when you are evaluating an intervention in one participant (or a small number analyzed individually) and you can build a random assignment into the design before data collection — randomizing the intervention start point in a phase design, the condition order in an alternating-treatments design, or the staggered onsets in a multiple-baseline design. It is the appropriate inferential tool when you want a defensible p-value to accompany visual analysis but cannot satisfy the normality and independence assumptions of conventional time-series statistics. It is not applicable to designs in which nothing was randomized — a fixed AB design with a predetermined phase change cannot support a randomization test — and it offers little power when the number of admissible assignments is very small, since the minimum p-value is the reciprocal of that number.
Strengths & limitations
- Provides an exact, distribution-free significance test whose validity depends only on the randomization built into the design, not on normality or independence.
- Directly suited to single-case and small-n disability research where group designs are infeasible and data are autocorrelated.
- Flexible in the choice of test statistic, allowing the analyst to target the specific predicted pattern of change.
- Complements visual analysis with a formal probability statement, strengthening the evidentiary value of single-case studies.
- Requires that an experimentally controllable feature be randomized before data collection; observational or fixed-phase designs cannot use it.
- Statistical power is bounded by the number of admissible assignments, so short series or few randomization opportunities yield weak tests.
- The test addresses only whether an effect exists for this case and does not by itself license generalization to a population.
- Computing the full permutation distribution can be demanding for complex designs, often requiring Monte Carlo approximation of the reference set.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
Why can a randomization test give a valid p-value without assuming normality?
Because the probability statements come from the design, not from the data's distribution. Edgington showed that when the experimenter randomly assigns an experimental feature — such as the intervention start point — the set of possible assignments is known and each is equally likely under the null hypothesis. The p-value is simply the proportion of those equally likely assignments that would yield a result as extreme as the observed one. Since this argument never invokes the shape of the data distribution, the test remains valid even when single-case data are non-normal and serially dependent.
What limits the statistical power of a single-case randomization test?
The number of admissible random assignments. The smallest p-value the test can ever return is one divided by the number of possible assignments in the randomization scheme. If, for instance, the intervention could only start at one of five randomly chosen sessions, the minimum attainable p-value is 0.20, which can never reach a 0.05 threshold no matter how large the effect. Designers therefore build in many randomization opportunities — a wide range of possible phase-onset points or many randomly ordered sessions — to make a powerful test possible.
Does a significant randomization test mean the intervention will work for other people?
No. The test establishes, for this particular case, that the observed pattern is unlikely to have arisen by chance under the design's randomization. It is an internally valid statement about one participant. Generalization to other individuals requires replication across cases, ideally combined through meta-analytic or multilevel synthesis. Edgington and Onghena are explicit that the randomization test answers the question of effect for the case at hand, leaving external validity to be established by accumulating replications.
Sources
- 1.Edgington, E. S. (1980). Validity of Randomization Tests for One-Subject Experiments. Journal of Educational Statistics, 5(3), 235-251.
- 2.Edgington, E. S., & Onghena, P. (2007). Randomization Tests (4th ed.). Chapman & Hall/CRC.ISBN 9781584885894
You have read it. What now?
Cite this page
ScholarGate. (2026, June 23). Randomization Test for Single-Case Designs. ScholarGate. https://scholargate.app/disability-studies/randomization-test-single-case