Quasi-Experimental and Natural Experiment Design

Quasi-experimental designs evaluate the effect of an intervention or exposure when random allocation is impossible, unethical, or impractical, using structured but non-random variation to approximate a controlled comparison. Natural experiments are a closely related form in which the exposure arises from a policy change, programme, or external event rather than from researcher assignment, letting investigators study population-level interventions that could never be randomized.

Definition

A quasi-experimental design estimates the effect of an intervention without random assignment, relying on a structured comparison (over time, between groups, or across a threshold) to isolate the intervention's effect; a natural experiment is a quasi-experiment in which the exposure is determined by events outside the investigator's control.

Scope

This topic covers the logic shared by quasi-experiments and natural experiments, common designs such as interrupted time series, controlled before-after studies, difference-in-differences, and regression discontinuity, and how they sit between randomized trials and purely observational studies. It is a methodological reference within evidence-based practice and offers no clinical instructions.

Core questions

• How can causal effects be estimated when randomization is not possible?
• What comparison structures do interrupted time series, difference-in-differences, and regression discontinuity exploit?
• What threats to validity remain when allocation is not random?

Key concepts

• Non-random assignment
• Interrupted time series
• Difference-in-differences
• Regression discontinuity
• Controlled before-after study
• Counterfactual and comparison group
• Confounding and secular trends

Mechanisms

Lacking randomization, these designs construct a counterfactual from structure rather than chance. Interrupted time series compares the trend before an intervention with the trend after, using the pre-intervention trajectory as the expected course absent the intervention (Kontopantelis et al., 2015). Difference-in-differences contrasts change over time in an exposed group with change in an unexposed comparison group to net out shared secular trends, and regression discontinuity exploits a threshold rule that assigns exposure to estimate effects near the cutoff. Because assignment is not random, residual confounding, history effects, and selection remain threats that must be addressed by design and analysis (Shadish et al., 2002).

Clinical relevance

Quasi-experiments and natural experiments provide much of the evidence on population-level and policy interventions in health, where randomization is often impossible. This entry explains how such evidence is generated and appraised and is not a basis for individual clinical decisions.

Evidence & guidelines

Medical Research Council guidance frames how natural experiments can credibly evaluate population health interventions and what conditions strengthen their inferences (Craig et al., 2012). Methodological accounts describe interrupted time series and related regression-based approaches when randomization is not an option (Kontopantelis et al., 2015), and in grading frameworks such designs are generally treated as observational evidence that may be upgraded when the comparison is strong (Guyatt et al., 2008).

History

The conceptual foundations were laid by Campbell and colleagues in mid-twentieth-century social science, distinguishing experimental from quasi-experimental designs and cataloguing threats to validity, later consolidated by Shadish, Cook, and Campbell (2002). Health research increasingly adopted these designs and natural experiments to evaluate policies and programmes, with dedicated guidance emerging for population health (Craig et al., 2012).

Debates

How much causal weight can quasi-experimental evidence bear?

Strong designs such as well-conducted natural experiments and regression discontinuity can approach the credibility of trials for certain questions, but without randomization the assumptions needed for causal inference are stronger and harder to verify, so the evidential weight is debated and design-dependent.

Key figures

• Donald Campbell
• Thomas Cook
• William Shadish
• Peter Craig

Seminal works

• shadish-2002
• craig-2012-natural
• kontopantelis-2015-its

Frequently asked questions

How does a quasi-experiment differ from a randomized trial?

Both evaluate an intervention against a comparison, but a quasi-experiment does not allocate the intervention at random; it relies on structure such as timing, group differences, or a threshold to build the comparison, which leaves more room for confounding.

What is a natural experiment?

It is a quasi-experiment in which the exposure is created by an event, policy, or programme outside the researcher's control, allowing study of interventions, often at population scale, that could not be assigned randomly.