Propensity Score Matching in Education

The key assumption is strong ignorability (no unmeasured confounding): conditional on the observed covariates, treatment assignment is independent of the potential outcomes. In plain terms, you must have measured all the variables that jointly affect both whether a unit is treated and its outcome. This assumption is untestable, which is the method's fundamental limitation; matching balances observed covariates but can do nothing about confounders you did not measure, so the credibility of any estimate rests on having captured the real drivers of selection.

By checking covariate balance, not by the propensity model's fit. After matching or weighting, compare the distributions of each covariate between treated and comparison groups, typically using standardized mean differences (a common rule of thumb is below 0.1) and inspecting distributions. If important covariates remain imbalanced, the matching has failed and should be revised. Good balance on observed covariates is the operational goal; a propensity model can predict treatment well yet still leave imbalance, so balance is the decisive diagnostic.

No. Randomization balances both observed and unobserved characteristics in expectation; propensity score matching balances only the covariates you measured. When selection is driven entirely by observed variables and overlap is good, matching can approximate experimental results, but when unmeasured confounders matter, it can leave meaningful bias. Evidence standards such as the What Works Clearinghouse rate well-executed matching designs below randomized trials, and sensitivity analyses are recommended to assess how robust conclusions are to hidden bias.