Robust Measurement Invariance Testing
Also known as: robust MI testing, robust measurement equivalence, non-normal measurement invariance, robust multi-group CFA invariance
Robust measurement invariance testing evaluates whether a psychometric instrument measures the same latent construct in the same way across groups when observed data violate multivariate normality. It adapts standard multi-group CFA sequences by replacing ordinary chi-square statistics with robust alternatives such as the Satorra-Bentler scaled statistic, yielding trustworthy conclusions about factor loadings, intercepts, and residual variances even with skewed or ordinal data.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use robust measurement invariance testing whenever you plan to compare latent means or structural paths across groups and your indicators are ordinal (e.g., Likert-type) or the data show marked skewness or excess kurtosis. Standard (ML) invariance tests should be replaced by their robust counterparts as a default in applied psychometric work because Likert scales virtually never satisfy multivariate normality. Robust MI testing is not appropriate when sample sizes per group are very small (fewer than roughly 100 per group), which makes estimation of the asymptotic covariance matrix unreliable. When all indicators are treated as continuous and multivariate normality is plausible, standard ML-based MI testing may be sufficient.
Strengths & limitations
- Corrects chi-square inflation under non-normality, dramatically reducing false rejections of adequate models.
- Follows the same well-understood configural-metric-scalar hierarchy as standard MI testing, so results are interpretable in the same framework.
- Compatible with widely available software (lavaan in R, Mplus) through well-documented options.
- Permits valid latent mean comparisons and structural path comparisons across groups even with ordinal or skewed data.
- Partial invariance solutions remain available, allowing researchers to salvage partial comparability when full scalar invariance fails.
- Requires large samples per group because accurate estimation of the asymptotic covariance matrix of moments is sample-intensive.
- The corrected difference test for nested model comparison is more complex to compute and report than the simple chi-square difference used under ML.
- Does not resolve the underlying measurement problem if items are genuinely non-invariant — it only ensures the test statistic is calibrated correctly.
- Results depend on the chosen estimator (e.g., MLR, WLSMV for ordinal data), and different robust estimators can yield somewhat different conclusions.
Frequently asked
What is the difference between robust measurement invariance testing and standard measurement invariance testing?
Both follow the same configural-metric-scalar hierarchy, but standard MI testing uses ordinary ML chi-square statistics that assume multivariate normality. Robust MI testing replaces those statistics with Satorra-Bentler scaled values and uses a corrected difference test for nested model comparisons, making conclusions trustworthy when data are skewed, kurtotic, or ordinal.
Can I use WLSMV instead of MLR for robust MI testing with Likert items?
Yes. WLSMV (diagonally weighted least squares with mean and variance adjustment) is often preferred for truly ordinal indicators because it models polychoric correlations rather than treating Likert responses as continuous. MLR is preferred when indicators are treated as approximately continuous despite non-normality. Both require the specialized difference test (DIFFTEST in Mplus; lavaan's lavTestScore or lavTestLRT with method='satorra.2000') rather than a naive chi-square subtraction.
What if full scalar invariance fails?
Test for partial scalar invariance: constrain the intercepts of a subset of items (at least two per factor) and free the non-invariant ones. If at least two intercepts per factor remain constrained, latent mean comparisons are still possible, but you must report the partial nature of invariance and interpret results cautiously.
How large must each group sample be?
There is no universal minimum, but the asymptotic covariance matrix required for the SB correction becomes unreliable with small samples. Many methodologists suggest at least 100–200 cases per group as a practical lower bound. With very small groups, Bayesian approaches to invariance testing may be more stable.
Which software can run robust MI tests?
Mplus supports robust estimators (MLR, WLSMV) with the DIFFTEST option for nested model comparisons. The R package lavaan implements MLR by default and provides the correct scaled difference test via lavTestLRT. The semTools package in R adds convenient wrapper functions for the full invariance testing sequence under robust estimation.
Sources
- Satorra, A. & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 399–419). Sage. link ↗
- Millsap, R. E. (2011). Statistical approaches to measurement invariance. Routledge. ISBN: 978-0805864786
How to cite this page
ScholarGate. (2026, June 3). Robust Measurement Invariance Testing. ScholarGate. https://scholargate.app/en/psychometrics/robust-measurement-invariance
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Confirmatory factor analysisPsychometrics↔ compare
- Measurement InvariancePsychometrics↔ compare
- SEMStatistics↔ compare