Hierarchical Confirmatory Research — Confirmatory Design for Nested Data
Hierarchical Confirmatory Research Design · Also known as: multilevel confirmatory research, nested confirmatory design, hierarchical hypothesis-testing research, HCR
Hierarchical confirmatory research is a quantitative design that tests pre-specified hypotheses about relationships or group differences in data that have a natural nested (hierarchical) structure — such as students clustered within classrooms, patients within hospitals, or employees within organizations. By explicitly modeling the hierarchy, it avoids the inflation of Type I error that occurs when nested data are analyzed as though observations were independent.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use hierarchical confirmatory research when you have theory-derived hypotheses about nested or clustered data — students in schools, patients in wards, employees in firms, repeated measures within persons — and the ICC indicates non-trivial between-group variance. It is the correct design when standard regression or ANOVA would violate independence assumptions and when the research goal is hypothesis testing rather than exploration. Do not use it as an exploratory fishing expedition with post-hoc hypothesis selection; the confirmatory logic requires hypotheses fixed before fitting. Also avoid it when cluster sizes are very small (fewer than 5–10 units per cluster) or when the number of higher-level units is too few (fewer than ~30 level-2 units) to estimate random effects reliably.
Strengths & limitations
- Accurately partitions variance between levels and provides unbiased standard errors for nested data.
- Enables testing of cross-level interaction hypotheses that single-level models cannot address.
- Maintains the confirmatory rigor of pre-specified hypothesis testing within a multilevel framework.
- Handles unbalanced group sizes and missing data at level 1 more robustly than repeated-measures ANOVA.
- Produces effect-size estimates at each level, enabling nuanced interpretation of where variance is explained.
- Requires adequate sample sizes at every level; small numbers of higher-level units (< 30) produce unreliable variance component estimates.
- Model specification demands strong theoretical grounding — misspecifying the level at which a variable operates leads to incorrect conclusions.
- More complex to implement and interpret than standard regression, requiring specialized software and multilevel modeling expertise.
- The confirmatory commitment means exploratory re-specification after seeing results inflates Type I error if not preregistered or cross-validated.
Frequently asked
How is hierarchical confirmatory research different from ordinary confirmatory research?
Ordinary confirmatory research tests pre-specified hypotheses assuming independent observations. Hierarchical confirmatory research does the same but explicitly models the clustering of observations within higher-level units. This is necessary when independence is violated — for example, students share classrooms — because ignoring clustering underestimates standard errors and inflates Type I error rates.
What ICC value justifies using a multilevel model?
A commonly cited threshold is ICC > .05, meaning more than 5% of total variance resides at the group level. Some methodologists suggest that even smaller ICCs can matter with large sample sizes. The ICC should always be reported; if it is negligible and cluster sizes are similar, a single-level confirmatory model may be sufficient.
Should I use ML or REML estimation?
Use REML (restricted maximum likelihood) when comparing models that differ in their random effects, as REML provides unbiased estimates of variance components. Use full ML when comparing models that differ in their fixed effects via likelihood ratio tests, because REML likelihood values are not comparable across models with different fixed effects.
Can I combine hierarchical confirmatory research with structural equation modeling?
Yes — multilevel structural equation modeling (MSEM) extends the hierarchical confirmatory framework to latent variable models, allowing confirmatory factor structures and path models at both within- and between-level units simultaneously. Software such as Mplus supports MSEM. This combination is appropriate when constructs are measured with multiple indicators and hypotheses involve latent relationships across levels.
How many participants do I need at each level?
A common heuristic is the 30/30 rule: at least 30 groups at level 2 with at least 30 individuals per group. However, requirements vary with the complexity of the model and the size of expected effects. Fewer level-2 units (as few as 10–20) may be acceptable for simple models with large fixed effects, but random effects estimates become unreliable with very few groups. A priori power analysis for multilevel designs is strongly recommended.
Sources
- Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods (2nd ed.). Sage. ISBN: 978-0761919049
- Hox, J. J. (2010). Multilevel Analysis: Techniques and Applications (2nd ed.). Routledge. ISBN: 978-1848728462
How to cite this page
ScholarGate. (2026, June 3). Hierarchical Confirmatory Research Design. ScholarGate. https://scholargate.app/en/research-design/hierarchical-confirmatory-research
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Confirmatory factor analysisPsychometrics↔ compare
- Confirmatory ResearchResearch Design↔ compare
- Hierarchical Model Testing ResearchResearch Design↔ compare
- Multilevel ModelingResearch Statistics↔ compare
- Structural Equation ModelingResearch Statistics↔ compare