Pilot Multi-Arm Experiment — Feasibility Testing Across Multiple Treatment Arms
Pilot Multi-Arm Experimental Design · Also known as: pilot multi-arm trial, feasibility multi-arm study, pilot parallel-arm experiment, preliminary multi-arm experiment
A pilot multi-arm experiment is a small-scale preliminary trial that tests the feasibility, logistics, and parameter estimates needed to plan a full-scale multi-arm study. It simultaneously evaluates two or more active treatment arms alongside a control, providing early evidence on recruitment rates, retention, protocol adherence, variability, and likely effect sizes before committing to a resource-intensive definitive experiment.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use a pilot multi-arm experiment when you plan a definitive multi-arm study but face substantial uncertainties about feasibility — recruitment, retention, protocol adherence, measurement instruments, or plausible effect sizes across arms. It is especially appropriate when each arm involves a distinct active treatment (not just dosage variation), when participant burden may vary across arms, or when the study setting is novel. Do not use it as a substitute for a properly powered efficacy trial: if your goal is to compare treatments definitively, design the full trial from the outset. Also avoid labelling an underpowered efficacy study as a 'pilot' post-hoc to excuse non-significant results.
Strengths & limitations
- Simultaneously tests the logistics of multiple arms, revealing arm-specific adherence or dropout issues early.
- Provides variance estimates and preliminary effect-size signals that improve the precision of full-trial power calculations.
- Allows refinement of the randomization and blinding procedure across all arms before large-scale commitment.
- Identifies the weakest or most burdensome arm before it can compromise a costly full trial.
- Encourages pre-specification of feasibility success criteria, promoting transparent decision-making.
- Too small to provide reliable efficacy conclusions; reporting effect estimates as if they were definitive results is a common and serious misuse.
- Each additional arm increases the complexity of logistics and coordination even at small scale.
- If the pilot deviates significantly from the planned full trial, lessons learned may not transfer directly.
- Recruitment and retention rates observed in a pilot may not reflect the full trial if the pilot context differs (e.g., pilot sites are more enthusiastic).
Frequently asked
How many participants do I need per arm in a pilot multi-arm experiment?
There is no single rule, but 10–30 participants per arm is a common pragmatic range. The goal is not statistical power for efficacy but enough observations to estimate variability, test data-collection procedures, and evaluate feasibility indicators. Justification should be based on the precision needed for the variance estimate rather than on detecting treatment differences.
Can I use the pilot data in the full trial analysis?
This depends on whether the pilot protocol was identical to the full trial and whether participants were blinded to any changes. If the pilot and full trial are seamless — same protocol, same sites, and a pre-specified internal pilot design — data can sometimes be pooled. In most external pilot designs, pooling is not appropriate because protocol changes between phases violate the assumption of homogeneous data.
What is the difference between a pilot multi-arm experiment and a multi-arm multi-stage (MAMS) design?
A pilot multi-arm experiment is a stand-alone feasibility study that precedes a full trial. A MAMS design is itself the full definitive trial, structured to drop underperforming arms at interim analyses using a pre-specified selection rule. MAMS trials are powered for efficacy; pilot multi-arm experiments are not.
Should I report effect sizes from the pilot?
You may report preliminary effect-size estimates with appropriate uncertainty (confidence intervals), but these must be framed explicitly as inputs to full-trial planning — not as evidence of efficacy or superiority of one arm over another. The estimates are highly unstable at pilot sample sizes.
Do I need ethics approval for the pilot and again for the full trial?
In most jurisdictions, yes. Ethics committees treat the pilot and the full trial as separate studies if there are protocol changes between them. Some approve a pilot with a condition that the full trial amendment is reviewed after the pilot results are available. Check with your institutional review board or ethics committee early in the planning process.
Sources
- Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P., Robson, R., Thabane, M., Giangregorio, L., & Goldsmith, C. H. (2010). A tutorial on pilot studies: The what, why and how. BMC Medical Research Methodology, 10(1), 1. DOI: 10.1186/1471-2288-10-1 ↗
- Chow, S.-C., & Liu, J.-P. (2008). Design and Analysis of Clinical Trials: Concepts and Methodologies (2nd ed.). Wiley-Interscience. ISBN: 978-0471249858
How to cite this page
ScholarGate. (2026, June 3). Pilot Multi-Arm Experimental Design. ScholarGate. https://scholargate.app/en/experimental-design/pilot-multi-arm-experiment
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Adaptive Multi-Arm ExperimentExperimental design↔ compare
- Factorial ExperimentExperimental design↔ compare
- Multi-arm experimentExperimental design↔ compare
- Pilot Field ExperimentExperimental design↔ compare
- Pilot Randomized Controlled TrialExperimental design↔ compare
- Randomized Controlled TrialExperimental design↔ compare