Testlet Response Theory

A testlet is a bundle of items that share a common stimulus, such as several questions tied to one reading passage. Standard IRT assumes that, conditional on ability, responses are independent. But items in a testlet draw on the same passage, so an examinee's responses to them are correlated beyond what ability explains — local item dependence. Ignoring this redundancy makes the test appear more informative and reliable than it is and can bias item and ability estimates, which is exactly what testlet response theory corrects.

They are related ways of relaxing conditional independence. A testlet model can be seen as a constrained multidimensional or bifactor model in which each testlet defines a narrow, nuisance dimension that affects only its own items, layered on the general ability dimension. Full multidimensional or bifactor IRT is more flexible but heavier; the testlet model is a parsimonious special case tailored to the common situation of shared-stimulus bundles. See the related Multidimensional Item Response Theory entry.

When the estimated testlet variances are negligible — that is, the items within bundles behave essentially independently given ability. Short testlets or stimuli that exert little common influence may induce trivial dependence, in which case standard IRT is simpler and adequate. The way to know is to fit the testlet model (or examine residual dependence) and check whether the testlet variances are meaningfully different from zero; only then is the added complexity warranted.