Pilot Solomon Four-Group Design — Experimental Design Feasibility Study
Also known as: Pilot S4GD, Solomon four-group pilot study, pilot four-group pretest-posttest control design, pilot SFGD
The Pilot Solomon Four-Group Design is a small-scale, preliminary implementation of the Solomon four-group experimental design. Its purpose is to test the feasibility and logistics of the full design before committing to a resource-intensive main study. The Solomon four-group design, introduced by Richard L. Solomon in 1949, controls for pretest sensitisation by using four groups — two that receive a pretest and two that do not — crossed with treatment and control conditions. Piloting this design allows researchers to estimate effect sizes, detect procedural problems, and verify that the pretest does not unduly influence posttest scores.
Key highlights
- Identifies and quantifies pretest sensitisation before it contaminates the main study, enabling better instrument selection.
- Provides realistic estimates of attrition, effect size, and variance needed for accurate main-study power calculations.
- Tests all procedural elements — randomisation, blinding, instrument administration, and intervention delivery — in a controlled, low-stakes environment.
- Produces a structured feasibility decision point that protects the main study investment from foreseeable procedural failures.
- Aligns with open-science norms by pre-registering the pilot and its role in the main study design.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use a Pilot Solomon Four-Group Design when you plan a main study using the Solomon four-group design and need to verify feasibility before full deployment — especially when the intervention is novel, the measurement instruments are untested in the target population, or participant attrition across four groups is a concern. It is particularly valuable when the research question requires distinguishing pretest sensitisation from genuine treatment effects. Do not use this design if you cannot recruit enough participants even for a small-scale pilot (fewer than 8 per cell makes estimates unreliable), if the intervention cannot be withheld from half the participants for ethical reasons, or if a simpler two-group pilot design is sufficient to answer your feasibility questions.
Strengths & limitations
- Identifies and quantifies pretest sensitisation before it contaminates the main study, enabling better instrument selection.
- Provides realistic estimates of attrition, effect size, and variance needed for accurate main-study power calculations.
- Tests all procedural elements — randomisation, blinding, instrument administration, and intervention delivery — in a controlled, low-stakes environment.
- Produces a structured feasibility decision point that protects the main study investment from foreseeable procedural failures.
- Aligns with open-science norms by pre-registering the pilot and its role in the main study design.
- Requires a substantially larger participant pool than a two-group pilot, increasing recruitment burden and cost.
- Small cell sizes in the pilot may yield imprecise pretest-sensitisation estimates, potentially misleading the power calculation.
- The pilot sample must not overlap with the main study, which can deplete available participants in small or specialised populations.
- Logistical complexity — coordinating four groups with and without pretests — increases risk of protocol deviations that may not occur at scale.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How is a pilot Solomon four-group design different from just piloting a pretest-posttest design?
A standard pretest-posttest pilot tests only two groups — treatment and control — both with pretests. It cannot detect whether the pretest itself is influencing posttest scores. The Solomon four-group pilot specifically adds two no-pretest groups, allowing the researcher to estimate and separate pretest sensitisation effects before those effects contaminate the main study results.
How many participants do I need per cell in the pilot?
A common recommendation is 10–15 participants per cell to obtain useful feasibility estimates, though some methodologists accept as few as 8. With fewer than 8 per cell the variance estimates are too unstable to inform main-study power calculations reliably. The pilot is not powered to detect the treatment effect itself — it is powered to estimate variance, attrition, and sensitisation effect magnitude.
Can I use the pilot data in my main study analysis?
No. Pilot participants must be excluded from the main study dataset. Including them inflates the effective sample size in ways not accounted for by the pre-registered power calculation and introduces potential bias if procedural adjustments were made between the pilot and main study. Report pilot results separately and transparently.
What statistical analysis should I run on the pilot data?
At minimum, run a 2×2 ANOVA on the posttest scores with pretest presence (yes/no) and treatment condition (treatment/control) as factors. The interaction term estimates the pretest sensitisation effect. Also compute descriptive statistics for each cell, estimate effect size (Cohen's d or eta-squared), and calculate observed variance for the main-study power calculation.
When should I decide that the Solomon four-group design is too complex for my study?
If the pilot reveals that maintaining four distinct groups causes severe ethical problems (treatment cannot be withheld), that attrition in the no-pretest groups is unacceptably high, or that the available population cannot sustain four separate cells in the main study, consider simplifying to a two-group randomised design with a sensitivity analysis for pretest effects. The pilot's job is precisely to surface this kind of decision.
Sources
- 1.Solomon, R. L. (1949). An extension of control group design. Psychological Bulletin, 46(2), 137–150.
- 2.Braver, M. C. W., & Braver, S. L. (1988). Statistical treatment of the Solomon four-group design: A meta-analytic approach. Psychological Bulletin, 104(1), 150–154.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Pilot Solomon Four-Group Design. ScholarGate. https://scholargate.app/experimental-design/pilot-solomon-four-group-design