Adaptive Pretest-Posttest Experimental Design

In a standard pretest-posttest design the sample size, treatment arms, and analytic plan are fully fixed before data collection begins. In the adaptive version, pre-specified decision rules allow certain design elements — most commonly sample size or arm allocation — to be modified during the trial based on accumulating data, while maintaining control of type I error through an alpha-spending function.

Not if adaptations are pre-specified and implemented through an independent unblinded statistician or data monitoring committee. The bias risk arises only when modifications are decided post-hoc after the investigators have seen unblinded outcome data. Properly designed adaptive trials with locked analysis plans have been shown to preserve type I error control.

When the correlation between pretest and posttest is moderate to high, ANCOVA is more efficient than the simple difference score because it uses the pretest as a covariate rather than subtracted noise. When the correlation is near zero, the two approaches perform similarly. Most methodologists recommend ANCOVA as the default primary analysis for pretest-posttest designs.

Each additional interim look consumes some of the alpha budget. O'Brien-Fleming and Pocock spending functions accommodate multiple looks while maintaining an overall 5% type I error rate. Practically, one to three interim analyses are common; more than five are rare and require very conservative alpha spending that reduces power at interim stages.

Adaptation rules on their own can be applied to quasi-experimental pretest-posttest studies, but without randomization, confounding cannot be ruled out and the causal interpretation of any treatment effect is substantially weakened. The design is most defensible — and regulatory acceptance most likely — when randomization is maintained throughout.