Pilot Factorial Experiment — Preliminary Factorial Design
Pilot Factorial Experiment · Also known as: preliminary factorial study, pilot factorial design, small-scale factorial trial, feasibility factorial experiment
A pilot factorial experiment is a small-scale, preliminary study that employs a factorial structure to simultaneously vary two or more factors across a limited number of experimental units. Its purpose is not to deliver definitive conclusions but to estimate effect sizes, within-group variance, and factor interactions, and to test logistical feasibility before committing resources to a full-scale factorial experiment. It is widely used in behavioral sciences, engineering, agriculture, and clinical research as an essential planning step.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use a pilot factorial experiment when you intend to run a full factorial or fractional-factorial study but lack reliable prior estimates of effect size or within-group variance — conditions that make power analysis guesswork. It is especially valuable when multiple factors are being studied for the first time in combination, when recruitment feasibility is uncertain, or when a new measurement instrument has not been validated in the target population. It is not appropriate as a standalone inferential study: because sample sizes are small by design, pilot factorial results should not be reported as definitive tests of hypotheses. Avoid it when budget constraints prevent any follow-up study, or when an adequate published meta-analysis already supplies reliable variance estimates.
Strengths & limitations
- Provides empirically grounded variance and effect-size estimates for accurate power calculations in the main study.
- Simultaneously tests multiple factors and their interactions in a resource-efficient preliminary stage.
- Identifies practical barriers — recruitment difficulty, attrition, protocol violations — before they undermine the main experiment.
- Factorial structure in the pilot preserves the interaction-detection capability that single-factor pilots lack.
- Helps refine factor levels and measurement instruments to match the target population.
- Small sample sizes mean that pilot effect-size estimates are themselves imprecise; they should be treated as rough guides rather than true population values.
- The pilot consumes participants from the same pool as the main study, which can reduce overall power if pilot participants cannot be re-recruited.
- Factorial pilots with many factors can still be resource-intensive; with more than three factors a fractional or screening design is usually preferable.
- If the pilot prompts major design changes, its variance estimates may no longer be valid for the revised protocol.
Frequently asked
How large should the pilot factorial experiment be?
There is no single rule, but common guidance suggests at least 5–15 experimental units per cell to obtain stable variance estimates. With a 2×2 design this implies 20–60 total participants. The required size depends on expected variability: highly variable outcomes need larger pilot samples to yield usable variance estimates. Published guidelines by Browne (1995) and others suggest a minimum of 30 participants total for simple designs, but the factorial structure may require more to populate all cells adequately.
Can I report inferential statistics (p-values) from the pilot?
You can report them for descriptive completeness, but they should not be interpreted as evidence for or against the main hypothesis. Pilot studies are statistically underpowered for confirmatory testing; significant results in a pilot are unreliable and are well known to overestimate effect sizes (the 'winner's curse'). Report confidence intervals for effect sizes and variance estimates instead.
Should pilot factorial data be combined with main-study data?
Only if this was pre-specified in a registered protocol before the pilot was run. Ad hoc pooling after the pilot has informed design changes is problematic because the two datasets may come from different protocol versions, inflating Type I error and introducing bias.
When is a fractional factorial pilot better than a full factorial pilot?
When the number of factors is four or more, a full factorial pilot requires many cells and is impractical at small sample sizes. A resolution-III or resolution-IV fractional factorial allows main effects (and some two-factor interactions) to be estimated with far fewer runs. The trade-off is that some interaction effects are confounded (aliased), so this choice is appropriate for screening, not for estimating specific interactions.
How do I use pilot results to compute sample size for the main study?
Extract the within-cell variance estimate (s^2) and a preliminary effect-size estimate from the pilot. Plug these into a power analysis formula for the chosen statistical test (e.g., ANOVA F-test, t-test per contrast) at your target power (e.g., 0.80) and alpha level (e.g., 0.05). Because pilot effect-size estimates are uncertain, it is prudent to add a 20–30% buffer to the calculated n, or to use the lower bound of the confidence interval around the effect size as a conservative input.
Sources
- Montgomery, D. C. (2017). Design and Analysis of Experiments (9th ed.). Wiley. ISBN: 978-1119113478
- Box, G. E. P., Hunter, J. S., & Hunter, W. G. (2005). Statistics for Experimenters: Design, Innovation, and Discovery (2nd ed.). Wiley. ISBN: 978-0471718130
How to cite this page
ScholarGate. (2026, June 3). Pilot Factorial Experiment. ScholarGate. https://scholargate.app/en/experimental-design/pilot-factorial-experiment
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Randomized Controlled TrialExperimental design↔ compare
- Response Surface MethodologyExperimental design↔ compare