Policy Evaluation Propensity Score Weighting
Propensity Score Weighting for Policy Evaluation · Also known as: PSW policy evaluation, inverse probability weighting for policy, IPW policy evaluation, policy PSW
Policy evaluation propensity score weighting applies inverse-probability weighting to observational data to estimate the causal effect of a policy program. By reweighting participants and non-participants so they resemble a target population, it removes selection bias from voluntary or administratively allocated program assignment without requiring randomization.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use this approach when a policy or program was administered without randomization and you want to recover a causal effect from observational or administrative data, provided you have a rich set of pre-treatment covariates that plausibly captures selection into the program. It is appropriate for cross-sectional, panel, or repeated cross-sectional data with a clearly defined treatment indicator and continuous or binary outcomes. Minimum sample guidance: at least 100 treated units, with adequate overlap so no propensity scores are near 0 or 1. Do NOT use when unmeasured confounders are the primary driver of selection (an instrument or difference-in-differences design is preferable), when overlap is severely violated (many scores near 0 or 1, causing extreme weights), or when the treatment is continuous rather than binary (marginal structural models are better suited).
Strengths & limitations
- Adjusts for a large number of observed confounders simultaneously using a single scalar, making it practical for high-dimensional covariate sets.
- Allows explicit targeting of policy-relevant estimands (ATE, ATT, ATU) that match the decision maker's question.
- Preserves the full sample — unlike matching, no observations are discarded, improving precision for large administrative datasets.
- Easily combined with outcome regression to form doubly robust estimators that remain consistent if either the propensity model or the outcome model is correctly specified.
- Transparent and auditable: the estimated propensity scores and covariate balance statistics can be reported alongside results for replication.
- Relies on the unconfoundedness assumption (selection on observables): if important confounders are unobserved, estimates remain biased regardless of how well the propensity model fits.
- Extreme weights caused by poor overlap between treated and control groups inflate variance sharply and make results sensitive to a handful of influential observations.
- Propensity model misspecification biases the weights; flexible methods (random forest, CBPS) mitigate but do not eliminate this risk.
- For very small treatment groups, weighted estimators can become unstable even with correct models.
- Standard errors require bootstrapping or sandwich estimators to account for the two-stage (score estimation + weighting) procedure.
Frequently asked
What is the unconfoundedness assumption and when does it hold?
Unconfoundedness (also called selection on observables) requires that, conditional on the observed covariates, treatment assignment is independent of potential outcomes. It holds when all variables that jointly influence both participation in the policy and the outcome have been measured. It is more plausible when administrative records include rich pre-treatment history, demographics, and prior outcomes.
Should I target ATE or ATT in a policy evaluation?
If you want to know the effect of universally rolling out the policy, estimate ATE. If you want to know what participants gained from it — which is typical for program assessment and accountability — estimate ATT. The choice changes the weighting formula and the interpretation of results.
What should I do if I have extreme weights?
First, check overlap: if propensity scores cluster near 0 or 1, the sample may not support the intended estimand. Options include trimming the sample to the region of common support, capping weights at a threshold (e.g., the 99th percentile), using stabilized weights, or redefining the target population to one where overlap is adequate.
How is this different from propensity score matching?
Matching discards unmatched units and estimates a local effect on the matched sample, while weighting retains the full sample and targets a pre-specified population estimand. Weighting is generally more efficient with large datasets; matching is more transparent and avoids extreme-weight problems but loses data.
Can I combine weighting with regression adjustment?
Yes — this yields a doubly robust estimator that is consistent if either the propensity model or the outcome regression model is correctly specified. This is generally recommended in applied policy evaluation to guard against partial model misspecification.
Sources
- Hirano, K., Imbens, G. W., & Ridder, G. (2003). Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score. Econometrica, 71(4), 1161-1189. DOI: 10.1111/1468-0262.00442 ↗
- Caliendo, M., & Kopeinig, S. (2008). Some Practical Guidance for the Implementation of Propensity Score Matching. Journal of Economic Surveys, 22(1), 31-72. DOI: 10.1111/j.1467-6419.2007.00527.x ↗
How to cite this page
ScholarGate. (2026, June 3). Propensity Score Weighting for Policy Evaluation. ScholarGate. https://scholargate.app/en/causal-inference/policy-evaluation-propensity-score-weighting
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Difference-in-DifferencesEconometrics↔ compare
- Doubly Robust EstimationCausal inference↔ compare
- Inverse Probability WeightingCausal inference↔ compare
- Policy Evaluation Difference-in-DifferencesCausal inference↔ compare
- Propensity Score MatchingResearch Statistics↔ compare
- Propensity Score WeightingCausal inference↔ compare