Survey Experiment

All three use randomization for causal inference, but differ in setting and sample. Lab experiments offer tight control but rely on small, often non-representative samples. Field experiments manipulate conditions in real-world settings with consequential outcomes but are costly and hard to control. Survey experiments embed the manipulation in a questionnaire administered to a (often representative) sample, balancing internal validity, generalizable samples, and low cost, at the price of brief, sometimes artificial stimuli.

Randomization makes the estimate internally valid for whoever is in the sample, but the magnitude and even sign of framing or information effects can vary across populations. A convenience sample of students may respond differently from the general electorate. Population-based samples let the estimated effect generalize to a defined target population, which is essential when the substantive claim concerns public opinion writ large rather than a particular subgroup.

Pre-treatment occurs when respondents have already been exposed in the real world to information similar to the experimental treatment before the study. Such respondents may already hold the attitude the treatment would induce, so the manipulation produces little additional movement, attenuating the measured effect. This makes experimental effects on heavily covered issues look weaker than the underlying causal force, a concern Gaines and colleagues emphasized.