Differential Item Functioning (DIF)

DIF operates at the individual item level and typically uses observed-score matching or IRT; measurement invariance testing (e.g., multi-group CFA) operates at the factor level and evaluates whether loadings, intercepts, and residuals are equal across groups. Both address the same underlying concern — that a scale measures the same construct equivalently across groups — but at different levels of analysis. Running both provides a more complete picture.

The Mantel-Haenszel procedure is simple, well-validated, and appropriate for dichotomous items when groups are large. Logistic regression is more flexible (handles polytomous items and tests for non-uniform DIF). IRT-based methods are preferred when an IRT model is already being fitted to the data and when precise ability estimation matters. Using at least two methods and requiring convergent flagging reduces false positives.

The reference group typically needs at least 200–500 examinees; the focal group at least 100–200, though larger is better. Very small focal groups (fewer than 50) have severely limited power to detect meaningful DIF, and results will be unstable. Sample-size requirements also increase for detecting non-uniform DIF.

A flagged item should be reviewed by content experts who evaluate whether the item contains construct-irrelevant content that disadvantages one group. If bias is confirmed, revise or remove the item. If the DIF reflects real group differences on a secondary dimension the item legitimately taps (impact rather than bias), the item may be retained but should be treated with care in score interpretation.

Yes. Logistic regression and polytomous IRT models (such as the graded response model or partial credit model) extend DIF analysis to ordinal polytomous items. The MH procedure in its basic form applies to dichotomous items only, but extensions for polytomous items exist.