Natural Experiment — Quasi-Experimental Causal Inference from Exogenous Events

The terms overlap but have a meaningful distinction. All natural experiments are quasi-experiments, but not all quasi-experiments are natural experiments. A quasi-experiment is any design lacking full randomization. A natural experiment is a specific subset in which the assignment mechanism is driven by a real-world event outside researcher control — the exogeneity of that event is the defining feature. Quasi-experiments may also include researcher-implemented non-random designs such as interrupted time series or matched comparison groups.

Exogeneity cannot be proven — it must be argued on theoretical grounds and supported empirically. The standard checks are: (1) balance tests showing no pre-treatment differences between groups on observable covariates; (2) placebo outcome tests showing the event has no effect on outcomes that should be unaffected; and (3) a credible narrative about why units could not anticipate or manipulate their assignment. If any of these checks fail, the design's credibility is undermined.

A small sample that passes balance and credibility checks is still valid, but statistical power will be limited. With few treated units, the variance of the treatment effect estimate is large, and small-sample inference corrections (e.g., randomization inference, clustered standard errors with few clusters) are necessary. If fewer than roughly 30 treated units are available, interpret results with caution and report confidence intervals rather than relying solely on p-values.

Often, but not always. Many natural experiments are implemented using instrumental variables when compliance with treatment is imperfect — the exogenous event serves as the instrument for actual treatment uptake. However, if the natural event assigns treatment perfectly (full compliance), a simple intent-to-treat comparison or a difference-in-differences estimator may suffice without formal IV machinery. The choice of estimator follows from the structure of the assignment mechanism.

Pre-treatment data are strongly recommended and often essential. Without them, balance checks — the primary empirical defense of the as-if randomization claim — cannot be conducted. If only post-treatment data exist, the researcher must rely entirely on theoretical arguments for exogeneity, which substantially weakens the design's credibility and peer review reception.