Factorial Pretest-Posttest Experimental Design

Both approaches are defensible, but ANCOVA with pretest as a covariate is generally preferred because it is statistically more powerful and does not assume equal measurement reliability across groups. Gain scores are simpler to compute and easier to communicate, but they are less efficient and can be misleading if pretest means differ substantially across groups. When randomisation succeeds, the two approaches converge; ANCOVA is recommended as the default.

Conduct a power analysis for the smallest effect you consider theoretically or practically meaningful — typically the interaction effect, which requires the largest N. Use a factorial ANOVA power analysis tool (e.g., G*Power) specifying the number of groups, the expected effect size (Cohen's f), desired power (commonly 0.80), and alpha level. As a rough guide, detecting a medium interaction effect (f = 0.25) in a 2 × 2 design with 80% power at alpha = .05 typically requires around 200 participants total (50 per cell).

A significant interaction means the effect of one factor depends on the level of the other. First, plot the cell means (an interaction plot) to visualise the pattern. Then conduct simple effects analyses — test the effect of Factor A separately at each level of Factor B (and vice versa). Apply an appropriate correction for multiple comparisons. Describe the interaction in plain language: for example, 'the inquiry method produced larger gains than the traditional method, but only in small classes; in large classes the methods did not differ.'

Yes. Three-factor (A × B × C) and higher-order factorial designs are mathematically straightforward, but they grow in complexity and sample-size demands rapidly. A three-factor design produces main effects, two-way interactions, and a three-way interaction. Three-way interactions are notoriously difficult to interpret and communicate. In practice, designs with more than three factors are rarely used in a single fully-crossed experiment; fractional factorial or other efficiency-optimised designs may be more practical.

The Solomon four-group design adds two groups that receive no pretest, allowing you to directly estimate whether the pretest itself influences posttest scores (the testing effect). If you have strong reason to believe that completing the pretest will sensitise participants to the treatment — for example, in attitude or awareness studies — the Solomon design is preferable. However, it requires roughly twice the sample size for the same statistical power, and it does not extend as naturally to factorial manipulations. For most factorial designs, the testing effect is minor and the gain in control from a Solomon structure rarely justifies the cost.