Multilevel Discriminant Validity
Also known as: multilevel DV, cross-level discriminant validity, hierarchical discriminant validity, ML-DV
Multilevel discriminant validity evaluates whether theoretically distinct constructs are empirically separable when data are nested within higher-level units such as teams, schools, or organizations. It extends single-level discriminant validity checks into a multilevel confirmatory factor analysis framework, verifying that constructs are distinguishable both within and between levels simultaneously.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use multilevel discriminant validity when your data are hierarchically structured (e.g., employees nested in teams, students nested in classrooms) and you need to demonstrate that conceptually distinct constructs are empirically separable at both individual and group levels. It is required in any multilevel scale validation study where constructs are theorized to operate or have meaning at more than one level. Do not apply it when data are not genuinely nested, when cluster sizes are very small (fewer than 5–10 units per group), or when the number of groups is too small (fewer than 30 groups) to reliably estimate between-group variance, as the between-level model will be unstable.
Strengths & limitations
- Simultaneously evaluates discriminant validity at both within- and between-group levels, capturing level-specific measurement quality.
- Prevents false validity conclusions that arise when single-level analyses ignore non-independence of observations.
- Compatible with established criteria (AVE, HTMT, chi-square difference test) applied at each level.
- Integrates naturally into a full multilevel CFA or multilevel SEM workflow.
- Reveals whether constructs that discriminate at the individual level also discriminate at the group level, an important theoretical check.
- Requires a sufficiently large number of groups (commonly recommended: at least 30–50 groups) to obtain stable between-level estimates.
- Small within-group sample sizes inflate within-level sampling error and reduce the reliability of within-level discriminant validity conclusions.
- Model complexity increases substantially relative to single-level CFA, and convergence failures are more common.
- AVE and HTMT thresholds were developed primarily for single-level models; their exact applicability at each level in ML-CFA is still debated in the literature.
- Software expertise and appropriate tools (e.g., Mplus, R lavaan with multilevel syntax) are required.
Frequently asked
Do I need to check discriminant validity at both levels?
Yes. In multilevel data, variance partitions into within-group and between-group components. A construct pair may discriminate adequately at one level but collapse at another. Both within-level and between-level discriminant validity must be evaluated and reported separately.
How many groups do I need for stable between-level estimates?
Simulation studies generally suggest at least 30 groups for basic multilevel models, with 50 or more preferred when the number of Level-2 parameters is large. Fewer groups lead to unstable between-level covariance estimates and unreliable discriminant validity conclusions at that level.
Can I use the HTMT criterion at the between level?
HTMT can be computed from the between-level correlation matrix, but the established simulation benchmarks (0.85, 0.90) were derived for single-level contexts. Treat between-level HTMT values as indicative rather than definitive, and supplement with chi-square difference testing.
What software can run multilevel discriminant validity analyses?
Mplus is the most flexible and widely used option, supporting ML-CFA with full model specification at both levels. R packages lavaan (with 'cluster' syntax) and OpenMx also provide multilevel SEM capabilities suitable for this purpose.
What is the difference between multilevel discriminant validity and measurement invariance?
Measurement invariance asks whether the same construct is measured equivalently across groups (equal loadings, intercepts). Discriminant validity asks whether two distinct constructs are empirically separable at each level. Both are needed for rigorous multilevel scale validation, but they address different validity questions.
Sources
- Dyer, N. G., Hanges, P. J., & Hall, R. J. (2005). Applying multilevel confirmatory factor analysis techniques to the study of leadership. Leadership Quarterly, 16(1), 149–167. DOI: 10.1016/j.leaqua.2004.09.009 ↗
- Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher's corner: Testing measurement invariance of second-order factor models. Structural Equation Modeling, 12(3), 471–492. DOI: 10.1207/s15328007sem1203_7 ↗
How to cite this page
ScholarGate. (2026, June 3). Multilevel Discriminant Validity. ScholarGate. https://scholargate.app/en/psychometrics/multilevel-discriminant-validity
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Confirmatory factor analysisPsychometrics↔ compare
- Discriminant ValidityPsychometrics↔ compare
- Multilevel CFAPsychometrics↔ compare
- Multilevel Convergent ValidityPsychometrics↔ compare
- Multilevel Measurement InvariancePsychometrics↔ compare