Ordinal Measurement Invariance Testing

Treating ordinal responses as continuous data underestimates correlations — especially for items with few categories or skewed distributions — and produces biased factor loadings, incorrect standard errors, and inflated chi-square statistics. Polychoric correlations with a categorical estimator such as WLSMV correct these biases and provide more accurate model fit and parameter estimates.

Both assess whether items perform equivalently across groups, but from different frameworks. DIF (from item response theory) focuses on individual items and individual response categories, testing whether the probability of a given response differs between groups after conditioning on the latent trait. Ordinal measurement invariance testing works within a CFA framework, constraining entire loading and threshold parameters across groups and evaluating overall model fit. DIF is more item-centric and granular; ordinal MI testing evaluates the whole scale structure simultaneously.

Partial invariance means that some — but not all — loadings or thresholds are equal across groups. Latent mean comparisons remain possible if at least two items per factor have invariant loadings and thresholds, provided the non-invariant items are clearly identified and their parameters freed. The interpretation of group differences is then qualified: only the construct as defined by the invariant items is being compared.

Mplus is the most widely used and supports WLSMV with the DIFFTEST option for model comparison. R packages lavaan (with estimator WLSMV and the lavTestScore/lavTestLRT functions) and semTools provide similar functionality. LISREL and OpenMx also support weighted least squares estimation for ordinal data.

Full scalar invariance is the ideal standard, but partial scalar invariance — with at least two items per factor having invariant thresholds — is generally accepted as a workable basis for latent mean comparisons, provided non-invariant items are reported. Some researchers also recommend checking that non-invariant thresholds are randomly rather than systematically distributed across groups to avoid biased mean estimates.