Cluster Randomized A/B Test
Also known as: cluster A/B test, group-randomized A/B test, network A/B test, cluster-level split test
A cluster randomized A/B test is an experimental design in which intact groups (clusters) — such as cities, schools, social network communities, or app user segments — are randomly assigned as whole units to either the treatment (A) or control (B) condition, rather than randomizing individual users or subjects. This approach is used when treatment effects would spill over between individuals if individual-level randomization were applied, or when the intervention must be delivered at the group level.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use a cluster randomized A/B test when (1) individual-level randomization would cause spillover or interference between treatment and control units — common in social networks, educational settings, or community health programs; (2) the intervention is inherently delivered at the group level (e.g., a school policy, a city-wide nudge, a platform algorithm applied to all users in a region); or (3) contamination between groups is a serious validity threat. Do NOT use it when clusters are too few (fewer than ~10 per arm makes valid inference very difficult), when individual randomization is feasible and spillover is negligible, or when the research question concerns individual-level mechanisms that get masked by cluster-level assignment.
Strengths & limitations
- Eliminates spillover and network interference that would bias estimates in individual-level A/B tests.
- Matches real-world deployment constraints when interventions are naturally applied at the group or community level.
- Randomization of clusters provides a valid basis for causal inference even when individuals within clusters are not exchangeable.
- Adaptable to both online digital experiments (geo-based holdouts) and offline field experiments (schools, clinics, villages).
- Stratified or matched-pair cluster assignment can improve balance on key covariates even with a modest number of clusters.
- Statistical power is substantially lower than individual-level randomization for a given total sample size — the design effect (DEFF = 1 + (m-1)*ICC) can be large when clusters are internally homogeneous.
- Requires a sufficient number of clusters (typically at least 10–15 per arm) to achieve reliable randomization balance and valid inference.
- Within-cluster treatment variation is impossible: everyone in a cluster receives the same arm, so individual heterogeneity in treatment response cannot be estimated.
- Defining cluster boundaries can be arbitrary or contentious, and poor boundary choices can reintroduce spillover.
Frequently asked
How many clusters do I need?
As a practical minimum, aim for at least 10–15 clusters per arm. Fewer clusters make it difficult to achieve balance through randomization, and standard asymptotic inference breaks down. With very few clusters (under 8 per arm), permutation tests or randomization inference are preferred over t-tests. Power calculations must use the design-effect formula: n_individual × (1 + (m-1) × ICC), where m is the average cluster size.
What is the intraclass correlation (ICC) and why does it matter?
The ICC measures how similar individuals within the same cluster are on the outcome, ranging from 0 (no clustering effect) to 1 (all variance is between clusters). A higher ICC means more redundancy within clusters and lower effective sample size. Even a small ICC — say 0.05 — can double or triple the required number of individuals if clusters are large. Always estimate or assume an ICC before designing a cluster randomized A/B test.
How is this different from a standard cluster RCT?
A cluster randomized A/B test is structurally identical to a cluster RCT but is typically distinguished by context (digital vs. clinical), speed (days/weeks vs. months/years), and the two-arm A/B framing common in industry. The statistical principles — cluster-level randomization, ICC adjustment, mixed-effects or GEE analysis — are the same in both settings.
Can I use this design to test multiple variants simultaneously?
Yes — extending to three or more arms (A/B/C or multi-arm) is straightforward: assign clusters to each arm randomly. This is sometimes called a multi-arm cluster randomized experiment. The minimum number of clusters per arm requirement applies to each arm, so the total cluster requirement grows with the number of arms.
What analysis method should I use?
Three approaches are common: (1) cluster-level summaries analyzed with a t-test or ANCOVA — simple but loses within-cluster information; (2) mixed-effects linear or logistic regression with a random intercept for cluster — flexible and handles covariates; (3) GEE with an exchangeable working correlation structure — robust to misspecification of the within-cluster correlation. All three are valid; choose based on sample size and whether you need individual-level covariate adjustment.
Sources
- Ugander, J., Karrer, B., Backstrom, L., & Kleinberg, J. (2013). Graph cluster randomization: Network exposure to multiple universes. Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 329–337. DOI: 10.1145/2487575.2487695 ↗
- Hayes, R. J., & Moulton, L. H. (2017). Cluster Randomised Trials (2nd ed.). CRC Press. ISBN: 9781498728874
How to cite this page
ScholarGate. (2026, June 3). Cluster Randomized A/B Test. ScholarGate. https://scholargate.app/en/experimental-design/cluster-randomized-ab-test
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Adaptive A/B testExperimental design↔ compare
- Blocked A/B TestExperimental design↔ compare
- Cluster Randomized Controlled TrialExperimental design↔ compare
- Factorial A/B TestExperimental design↔ compare
- Field ExperimentExperimental design↔ compare
- Multi-arm experimentExperimental design↔ compare