Policy Evaluation Inverse Probability Weighting
Inverse Probability Weighting for Policy Evaluation · Also known as: IPW policy evaluation, propensity-weighted policy analysis, inverse probability of treatment weighting
Policy evaluation inverse probability weighting (IPW) uses estimated propensity scores to reweight observed units so that the weighted sample mimics a randomised experiment. Each unit is weighted by the inverse of its probability of receiving the policy, creating a pseudo-population in which treatment assignment is independent of observed covariates and the average treatment effect (ATE) can be read off directly.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use policy evaluation IPW when you have observational data on a binary policy or programme, a rich set of pre-policy covariates that plausibly satisfies the ignorability (no unmeasured confounding) assumption, and you want to estimate the average treatment effect for the full population rather than only for the treated. It suits cross-sectional or pooled panel data with moderate to large samples (at least 100 units per arm is advisable). Do not use it when the propensity scores are near 0 or 1 for many units (extreme weights inflate variance severely), when important confounders are unobserved, or when treatment is continuous rather than binary without adaptation.
Strengths & limitations
- Directly targets the average treatment effect (ATE) for the full population, not just for the treated, which is often the relevant policy quantity.
- Flexible propensity-score model: logistic regression, random forests, or gradient boosting can all be used to estimate e(X).
- Separates the design stage (propensity estimation and balance checking) from the analysis stage, allowing the researcher to verify balance before examining outcomes.
- Compatible with doubly robust estimators that remain consistent if either the propensity or the outcome model is correctly specified.
- Applicable in both cross-sectional and panel settings and extendable to time-varying treatments via marginal structural models.
- Relies on the ignorability assumption (no unmeasured confounding): if important covariates are missing, the estimate is biased and there is no diagnostic to detect this.
- Extreme propensity scores (near 0 or 1) produce very large weights, inflating variance and making estimates unstable; stabilised or trimmed weights help but do not fully resolve the problem.
- Propensity-score model misspecification leads to poor covariate balance and biased estimates; the method is not self-validating.
- Efficiency is generally lower than doubly robust or regression-based methods when the outcome model is correctly specified.
- With very small samples the propensity model is estimated imprecisely, degrading balance and inference quality.
Frequently asked
What is the ignorability assumption and why does it matter?
Ignorability (also called unconfoundedness or selection on observables) states that, conditional on observed covariates, treatment assignment is as good as random. Without it, the IPW estimate conflates the policy effect with selection bias. It cannot be tested from the data, so subject-matter knowledge about what drives programme participation is essential.
How do I handle extreme propensity scores?
Units with scores very close to 0 or 1 receive enormous weights that destabilise the estimator. Common remedies include using stabilised weights, trimming observations with weights above a chosen percentile, or switching to a doubly robust estimator that is less sensitive to extreme scores.
What is the difference between ATE and ATT, and which does IPW give me?
The ATE is the average effect if the entire population were treated; the ATT is the average effect among those actually treated. Standard Horvitz-Thompson IPW estimates the ATE. To target the ATT, reweight only the untreated units by e(X)/(1-e(X)) and compare them to the treated group directly.
Should I use IPW or propensity-score matching for policy evaluation?
IPW uses all observations and targets the ATE for the full population; matching discards unmatched units and typically estimates the ATT. IPW is preferred when the policy question concerns population-wide effects and the propensity model fits well. Matching is more robust to extreme scores but loses efficiency when many units are discarded.
Can I use machine learning to estimate the propensity score?
Yes. Random forests, gradient boosting, and lasso logistic regression often improve covariate balance compared with standard logistic regression, especially with many covariates. However, using ML-estimated scores in a standard IPW estimator can introduce regularisation bias; cross-fitted doubly robust estimators (e.g., double machine learning) are preferable in that setting.
Sources
- Imbens, G. W., & Wooldridge, J. M. (2009). Recent Developments in the Econometrics of Program Evaluation. Journal of Economic Literature, 47(1), 5-86. DOI: 10.1257/jel.47.1.5 ↗
- Robins, J. M., Hernan, M. A., & Brumback, B. (2000). Marginal Structural Models and Causal Inference in Epidemiology. Epidemiology, 11(5), 550-560. DOI: 10.1097/00001648-200009000-00011 ↗
How to cite this page
ScholarGate. (2026, June 3). Inverse Probability Weighting for Policy Evaluation. ScholarGate. https://scholargate.app/en/causal-inference/policy-evaluation-inverse-probability-weighting
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Doubly Robust EstimationCausal inference↔ compare
- Inverse Probability WeightingCausal inference↔ compare
- Marginal Structural ModelCausal inference↔ compare
- Policy Evaluation Propensity Score MatchingCausal inference↔ compare
- Propensity Score MatchingResearch Statistics↔ compare
- Propensity Score WeightingCausal inference↔ compare