Multilevel Convergent Validity
Also known as: cross-level convergent validity, multilevel measurement validity, between-level convergent validity
Multilevel convergent validity evaluates whether items or scales intended to measure the same construct show coherent, strong associations at each level of a nested data structure — within individuals, within groups, and between groups. It extends classical convergent validity from single-level measurement models into the multilevel confirmatory factor analysis (ML-CFA) framework.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use multilevel convergent validity when (1) data are nested — individuals in groups, students in classrooms, employees in teams — and (2) you intend to interpret or aggregate the scale at the group level as well as the individual level. It is essential when reporting group-level analyses (e.g., team climate, school culture) that rely on survey items validated only at the individual level. Do NOT apply it when data are purely independent (no clustering), when cluster sizes are very small (fewer than five per group makes between-level estimation unreliable), or when you have fewer than about thirty groups — such designs have insufficient between-level power to estimate group-level factor loadings stably.
Strengths & limitations
- Provides rigorous evidence that a scale measures the intended construct at both individual and group levels, supporting multilevel theory.
- Separates within-group reliability from between-group reliability, exposing a scale's level-specific measurement quality.
- Aligns validity evidence with the level at which substantive hypotheses are tested, preventing level-of-analysis mismatches.
- Can detect constructs that are valid at one level but invalid at another, guiding targeted scale revision.
- Integrates naturally with multilevel SEM, allowing validity checks and structural hypothesis testing in one framework.
- Requires large numbers of groups (typically ≥ 30–50) for stable between-level estimation; many organizational datasets fall short.
- Very small within-group cluster sizes reduce within-level power and produce unstable loadings.
- Interpretation of the between-level factor requires a clear theoretical rationale for why the construct should vary across groups; validity evidence alone does not supply that rationale.
- Software setup (e.g., Mplus TYPE=TWOLEVEL CFA) is more complex than single-level CFA, increasing the risk of specification errors.
Frequently asked
How is multilevel convergent validity different from ordinary convergent validity?
Ordinary convergent validity examines whether indicators of the same construct correlate strongly in a single-level CFA. Multilevel convergent validity does this separately at each level of a nested design — within groups and between groups — using ML-CFA. The two analyses can yield different conclusions: a scale may show strong within-level loadings but near-zero between-level loadings, meaning it should not be used for group-level inferences.
What software can I use to run ML-CFA for this purpose?
Mplus is the most widely used tool; TYPE=TWOLEVEL CFA handles between- and within-level structures simultaneously. R packages lavaan (with cluster options) and nlme-based extensions also support multilevel CFA, though Mplus provides the most complete set of level-specific fit statistics.
What AVE threshold should I use at the between level?
The conventional threshold of AVE ≥ 0.50 applies at each level. In practice, between-level AVE is often harder to achieve because fewer data points drive the between-level estimates. Some researchers accept slightly lower values (≥ 0.40) at the between level provided loadings are significant and ICC(2) is adequate (≥ 0.70).
Does passing multilevel convergent validity mean I can freely aggregate scores to the group level?
It is a necessary but not sufficient condition. You also need evidence from ICC(1), ICC(2), and inter-rater agreement statistics (e.g., rwg) that meaningful between-group variance exists and that group means are reliable estimates of the group-level construct.
Can multilevel convergent validity be assessed with only two levels?
Yes. A two-level design — individuals nested in groups — is the most common application. Three-level designs (e.g., students in classrooms in schools) are theoretically possible but require substantially larger samples at every level and are rarely estimated in practice.
Sources
- Dyer, N. G., Hanges, P. J. & Hall, R. J. (2005). Applying multilevel confirmatory factor analysis techniques to the study of leadership. Leadership Quarterly, 16(1), 149–167. DOI: 10.1016/j.leaqua.2004.09.009 ↗
- Chen, G., Bliese, P. D. & Mathieu, J. E. (2005). Conceptual framework and statistical procedures for delineating and testing multilevel theories of homology. Organizational Research Methods, 8(4), 375–409. DOI: 10.1177/1094428105280056 ↗
How to cite this page
ScholarGate. (2026, June 3). Multilevel Convergent Validity. ScholarGate. https://scholargate.app/en/psychometrics/multilevel-convergent-validity
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Confirmatory factor analysisPsychometrics↔ compare
- Construct ValidityPsychometrics↔ compare
- Discriminant ValidityPsychometrics↔ compare
- Measurement InvariancePsychometrics↔ compare