Counterfactual Impact Evaluation (CIE)
Counterfactual Impact Evaluation · Also known as: CIE, counterfactual evaluation, counterfactual policy evaluation, impact evaluation
Counterfactual Impact Evaluation is a family of causal methods that estimates the effect of an intervention by comparing what actually happened to participants with what would have happened had the intervention not taken place. Formalised in the Rubin Causal Model and extended by Heckman, Imbens and others, CIE underlies most modern program and policy evaluation practice.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+6 more
When to use it
Use CIE whenever the research question is 'did this intervention cause a change in outcomes?' and randomisation is not feasible or did not occur. It applies across program evaluation, health economics, education research, and policy analysis, provided you can define a credible comparison condition and have data on both treated and comparison units before and/or after the intervention. Do not use CIE as a single method label for routine regression; it is a framework that selects and governs a specific identification strategy. Avoid CIE when the assignment mechanism is entirely unknown and no design feature or valid comparison group can be identified.
Strengths & limitations
- Provides a rigorous causal framework that makes the counterfactual assumption explicit rather than implicit.
- Covers a broad range of methods (RCT, DiD, RDD, IV, matching, SCM), allowing the analyst to choose the strategy best suited to the data and institutional context.
- Widely adopted in policy evaluation — particularly by the European Commission and World Bank — giving results institutional credibility.
- Forces the researcher to state and defend identification assumptions before running any model, reducing specification searching.
- Can be applied retrospectively to observational data when experimental evidence is unavailable.
- Each identification strategy within CIE rests on untestable or partially testable assumptions; violations produce biased estimates.
- Demands detailed knowledge of the assignment mechanism and substantial data on pre-treatment characteristics and outcomes.
- The framework itself does not choose the method — misapplying an identification strategy to a setting where its assumptions fail is a serious risk.
- External validity (generalisability to other populations or contexts) is not guaranteed even when internal validity is achieved.
Frequently asked
Is counterfactual impact evaluation a single statistical method?
No. CIE is a causal framework that encompasses multiple identification strategies — difference-in-differences, regression discontinuity, instrumental variables, matching, and synthetic control — each suited to different data structures and assignment mechanisms. The analyst must choose and justify the appropriate method for their setting.
What is the fundamental problem of causal inference that CIE addresses?
Each unit is observed in only one state — treated or untreated — so the counterfactual outcome is never directly observed. CIE makes this missing potential outcome explicit and uses design features or statistical adjustments to reconstruct it as credibly as possible.
How do I know which identification strategy to choose?
The choice depends on the assignment mechanism: if treatment was assigned at a sharp cutoff, use RDD; if there was a natural experiment affecting only some units over time, use DiD; if treatment is endogenous and you have a valid instrument, use IV; if selection is on observables and overlap is sufficient, use matching or weighting.
What is the difference between ATT and ATE?
The ATT (average treatment effect on the treated) is the average causal effect for the units that actually received the intervention. The ATE is the average effect across the full population. Most quasi-experimental methods identify the ATT; only under strong additional assumptions do they recover the ATE.
How should I validate the counterfactual?
Run placebo tests — apply the method to pre-treatment periods or to outcomes that should be unaffected by the intervention. Check balance in covariates between treated and comparison units. For DiD, test pre-treatment trend parallelism. For RDD, test for covariate discontinuities at the cutoff.
Sources
- Heckman, J. J., & Vytlacil, E. J. (2007). Econometric evaluation of social programs, Part I: Causal models, structural models and econometric policy evaluation. Handbook of Econometrics, 6B, 4779-4874. DOI: 10.1016/S1573-4412(07)06070-9 ↗
- Imbens, G. W., & Wooldridge, J. M. (2009). Recent developments in the econometrics of program evaluation. Journal of Economic Literature, 47(1), 5-86. DOI: 10.1257/jel.47.1.5 ↗
How to cite this page
ScholarGate. (2026, June 3). Counterfactual Impact Evaluation. ScholarGate. https://scholargate.app/en/causal-inference/counterfactual-impact-evaluation
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Causal Impact AnalysisCausal inference↔ compare
- Difference-in-DifferencesEconometrics↔ compare
- Instrumental Variables in Health ResearchHealth Economics↔ compare
- Propensity Score MatchingResearch Statistics↔ compare
- Synthetic Control MethodCausal inference↔ compare