Multilevel Confirmatory Factor Analysis (MCFA)

Ordinary multi-group CFA tests whether the same factor structure holds across known discrete groups (e.g., gender, country), treating group membership as a fixed moderator. MCFA treats groups as a random sample from a population of groups and simultaneously models the covariance structure at both the individual and the group level. The two approaches answer different questions: multi-group CFA asks whether parameters differ between specific groups; MCFA estimates the measurement structure at each level and partitions variance between levels.

No, and this is one of the key findings MCFA can produce. The within-level structure reflects how individual differences on latent traits drive item responses; the between-level structure reflects how group means on those constructs vary. These can differ. A common finding is that multiple within-level factors collapse to a single between-level factor, suggesting that the finer distinctions among constructs exist only at the individual level.

ICCs above roughly 0.05 are commonly taken as evidence of non-trivial clustering that warrants a multilevel approach. ICCs above 0.10 are clearly substantial. Very low ICCs (below 0.02–0.03 for all items) suggest that grouping has negligible effect, and single-level CFA may suffice, though formally testing this with a multilevel model is still best practice when data were collected in groups.

As a rough guideline, at least 30 groups with at least 10 observations each are needed for reasonably stable between-level estimates. With fewer than 20 groups, between-level parameters can be very imprecise. Simulation studies suggest that sample sizes at the between level are the binding constraint: having many individuals in few groups gives poor between-level estimation, whereas many groups — even with modest group sizes — improves it substantially.

Yes. Cross-level invariance testing examines whether the factor loadings at the between level equal those at the within level (a form of isomorphism). Additionally, standard multi-group MCFA can test across demographic or cultural groups nested within a higher-level structure. Demonstrating cross-level measurement invariance is a prerequisite for meaningfully aggregating individual scores to represent group-level constructs.