Robust Causal Impact Analysis
Robust Causal Impact Analysis with Sensitivity and Placebo Checks · Also known as: robust CausalImpact, sensitivity-augmented causal impact, causal impact with robustness checks, robust BSTS causal inference
Robust Causal Impact Analysis extends the Bayesian structural time-series CausalImpact framework (Brodersen et al., 2015) by embedding systematic robustness checks — in-time placebo tests, in-space placebo controls, covariate sensitivity analysis, and prior sensitivity assessments — to verify that a detected intervention effect is genuine and not an artifact of model choices or coincidental data patterns.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Robust Causal Impact Analysis when you have a single treated unit observed over a time series and want to guard against false positives caused by covariate selection or coincidental shocks. It is most valuable in high-stakes or policy-relevant applications where a single CausalImpact run might not be convincing to reviewers. Adequate pre-intervention data (typically at least 60 time points) and meaningful control series that co-move with the treated unit are required. Do not use it when you have very short pre-periods (fewer than 20 observations), when no control covariates are available, or when the intervention window is ambiguous — standard interrupted time-series or a simple DiD is more transparent in those cases.
Strengths & limitations
- Provides explicit evidence of credibility beyond a single model run, making causal claims more defensible in peer review and policy contexts.
- Inherits the full Bayesian uncertainty quantification of CausalImpact, giving posterior probabilities and credible intervals rather than only point estimates and p-values.
- Placebo tests reveal whether the detected effect is anomalously large relative to a data-driven null distribution, adding a nonparametric validation layer.
- Covariate and prior sensitivity checks expose fragility early, enabling better covariate selection before final reporting.
- Handles multivariate control series automatically through the spike-and-slab prior, which performs variable selection without manual specification.
- Requires a sufficiently long pre-intervention series (typically 60+ time points) for reliable BSTS parameter estimation; short series yield wide, uninformative posteriors.
- Multiple robustness runs increase computational cost substantially, especially with many candidate covariates or many placebo iterations.
- The approach is designed for a single treated unit; extending to multiple simultaneously treated units requires staggered or panel adaptations that are not trivial.
- Robustness checks reduce but do not eliminate the possibility of a false positive when unobserved confounders drive both the treated unit and the chosen control series.
Frequently asked
How many placebo iterations are needed for a meaningful rank test?
In-space placebo tests are most informative when the pool of untreated units is at least 20; this gives a minimum p-value of 0.05 (1/20) under the rank test. Fewer units make the test coarse. In-time placebo tests benefit from a long pre-period with multiple candidate placebo dates spread across distinct sub-windows.
What counts as a passing robustness check?
There is no universal threshold, but a common convention is that the estimated cumulative effect retains the same sign and remains statistically credible (posterior probability above 0.90 or 0.95) across all major perturbations of covariates and priors, and that the true treated unit ranks in the top 5-10% of the in-space placebo distribution.
Can Robust Causal Impact Analysis handle staggered interventions?
The standard framework targets a single clean intervention date. Staggered interventions across multiple units require panel-DiD extensions (e.g., Callaway-Sant'Anna or Synthetic DiD) rather than BSTS-based CausalImpact.
Is the 'robust' variant implemented in the standard CausalImpact R package?
The base CausalImpact package (Brodersen et al.) implements the core BSTS model. Robustness checks — especially in-space placebo loops and covariate sensitivity grids — must typically be scripted by the analyst. Some community extensions and proprietary tools automate these steps.
How does Robust Causal Impact Analysis differ from the Synthetic Control Method?
Both construct a counterfactual from donor units. The Synthetic Control uses a convex combination of donor units without a time-series model, while CausalImpact uses a BSTS model with donor series as covariates and allows the spike-and-slab prior to select among them. Robust CausalImpact adds systematic placebo and sensitivity checks that are analogous to, but distinct from, the inference-by-permutation approach standard in the Synthetic Control literature.
Sources
- Brodersen, K. H., Gallusser, F., Koehler, J., Remy, N., & Scott, S. L. (2015). Inferring causal impact using Bayesian structural time-series models. Annals of Applied Statistics, 9(1), 247-274. DOI: 10.1214/14-AOAS788 ↗
- Cunningham, S. (2021). Causal Inference: The Mixtape. Yale University Press. ISBN: 978-0300251685
How to cite this page
ScholarGate. (2026, June 3). Robust Causal Impact Analysis with Sensitivity and Placebo Checks. ScholarGate. https://scholargate.app/en/causal-inference/robust-causal-impact-analysis
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Causal Impact AnalysisCausal inference↔ compare
- Causal Impact AnalysisCausal inference↔ compare
- Interrupted Time SeriesCausal inference↔ compare
- Sensitivity Analysis for CausalityCausal inference↔ compare
- Synthetic Control MethodCausal inference↔ compare