Factorial Survey Experiment

Both cross many randomized attributes, but the response task differs. In a factorial survey, respondents rate each vignette on a scale, and analysis regresses ratings on attribute levels with a multilevel model. In a conjoint experiment, respondents typically choose between or rank profiles, and analysis estimates average marginal component effects from those choices. Conjoint's forced choice mimics real selection and reduces scale-use bias; the factorial survey's ratings preserve graded judgment and intensity.

Each respondent rates several vignettes, so the ratings are clustered within respondents and not independent. A multilevel model adds a respondent-level random effect to absorb that clustering and individual rating tendencies, producing correct standard errors for the dimension effects. It also lets the analyst model heterogeneity — whether vignette effects depend on respondent characteristics — through cross-level interactions, separating shared judgment norms from individual variation.

Full factorial crossing can generate combinations that are logically impossible or wildly unrealistic, which confuse respondents and add noise. Researchers handle this by imposing constraints that exclude impossible cells from the sampling universe, or by restricting level ranges so all combinations remain plausible. The trade-off is that constraints can reintroduce correlations among dimensions, so they are applied sparingly and documented.