Heterogeneous Treatment Effect Counterfactual Impact Evaluation
Also known as: HTE-CIE, heterogeneous CIE, CATE-based counterfactual evaluation, subgroup counterfactual impact evaluation
Heterogeneous Treatment Effect Counterfactual Impact Evaluation (HTE-CIE) extends standard counterfactual impact evaluation by estimating how the causal effect of a policy or intervention varies across subgroups defined by pre-treatment characteristics. Rather than reporting a single average treatment effect, it maps the Conditional Average Treatment Effect (CATE) across the covariate space, revealing who benefits most or least from an intervention.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use HTE-CIE when a policy or programme has been implemented on some units but not others and you suspect that its impact differs meaningfully across participant characteristics such as firm size, sector, age, or prior performance. It is appropriate with cross-sectional or panel data where both treated and comparable untreated units are observed, and where sample sizes are large enough to estimate heterogeneous effects reliably (typically n > 200 per subgroup). Do not use it when the overall sample is very small, when treatment assignment is fully endogenous without an identification strategy, or when the goal is only an aggregate average effect and subgroup differences are not substantively relevant.
Strengths & limitations
- Reveals treatment effect heterogeneity that is hidden in average estimates, enabling more precise policy targeting.
- Compatible with a wide range of identification strategies — matching, IPW, regression discontinuity, DiD — each extended to the CATE setting.
- Machine learning estimators such as causal forests handle high-dimensional moderators without requiring pre-specified interaction terms.
- Directly supports cost-effectiveness analysis by identifying subgroups with the highest return on intervention.
- Consistent with the European Commission counterfactual impact evaluation guidelines, making results actionable for public programme assessment.
- CATE estimation requires substantially larger samples than average effect estimation; underpowered studies yield imprecise and unreliable subgroup effects.
- Multiple subgroup comparisons raise the risk of spurious findings unless multiple-testing corrections or pre-registration are applied.
- The validity of all heterogeneity estimates still depends on the underlying identification assumption (parallel trends, conditional ignorability, or exclusion restriction) holding within each subgroup.
- Causal forest and related ML methods are computationally intensive and require careful tuning and cross-validation.
Frequently asked
How is HTE-CIE different from standard counterfactual impact evaluation?
Standard CIE estimates a single average treatment effect (ATE or ATT) for the whole sample. HTE-CIE goes further by estimating the Conditional Average Treatment Effect as a function of pre-treatment covariates, revealing whether and how impacts vary across subgroups.
What sample size do I need for reliable CATE estimates?
There is no universal threshold, but as a rule of thumb each subgroup of interest should contain at least 100-200 observations to produce reasonably precise CATE estimates. Machine learning methods such as causal forests require even larger samples to exploit high-dimensional moderators meaningfully.
Which estimator should I use for the CATE?
The choice depends on the identification strategy and sample size. Interacted regression is transparent and easy to implement. Causal forests (Athey & Wager) handle many moderators automatically. Double machine learning (DML) is preferred when both the outcome and treatment models are complex. All require a valid identification assumption (ignorability, parallel trends, or an instrument).
How do I avoid false-positive subgroup findings?
Pre-register subgroups of interest before analysis, apply Bonferroni or Benjamini-Hochberg corrections when testing multiple subgroups, and use cross-fitted or honest causal forest estimates which split sample for model fitting and inference to guard against overfitting.
Is HTE-CIE compatible with the European Commission evaluation guidelines?
Yes. The EC CIE guidelines explicitly recommend examining whether treatment effects differ across firm or participant characteristics. HTE-CIE formalises this recommendation using modern econometric and machine learning estimators.
Sources
- Cerulli, G. (2010). Modelling and measuring the effect of public subsidies on business R&D: A critical review of the econometric literature. Economic Record, 86(274), 421-449. DOI: 10.1111/j.1475-4932.2009.00615.x ↗
- Athey, S., & Wager, S. (2019). Estimating treatment effects with causal forests: An application. Observational Studies, 5(2), 37-51. link ↗
How to cite this page
ScholarGate. (2026, June 3). Heterogeneous Treatment Effect Counterfactual Impact Evaluation. ScholarGate. https://scholargate.app/en/causal-inference/heterogeneous-treatment-effect-counterfactual-impact-evaluation
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Counterfactual Impact EvaluationCausal inference↔ compare
- Heterogeneous Treatment Effect Difference-in-DifferencesCausal inference↔ compare
- Marginal Structural ModelCausal inference↔ compare
- Propensity Score MatchingResearch Statistics↔ compare