Pragmatic Screening Test Evaluation — Real-World Screening Performance

A standard (explanatory) diagnostic accuracy study maximises internal validity: strict eligibility, standardised test administration by trained research staff, mandatory reference-standard testing for all participants, and tightly supervised designs. A pragmatic screening evaluation deliberately relaxes these controls to reflect real-world conditions — broad eligibility, routine staff, opportunistic reference-standard ascertainment — so that measured accuracy predicts performance after deployment rather than under ideal research conditions.

PRECIS-2 is a nine-domain rating tool that helps researchers position their design on a spectrum from fully explanatory to fully pragmatic across dimensions such as eligibility, recruitment, adherence, follow-up, and outcome measurement. It is not mandatory, but using it strengthens transparency, helps peer reviewers assess the degree of pragmatism, and ensures the team makes conscious rather than accidental choices about each design element.

This is the partial verification problem, common in pragmatic designs. Options include: disease-free survival follow-up as a surrogate reference standard for screen-negatives; flow-adjusted analysis accounting for the verification fraction; multiple imputation of missing reference-standard results; and sensitivity analyses under best-case and worst-case assumptions. Always report the proportion verified and justify the chosen approach.

Yes — leveraging EHR data is a hallmark of pragmatic designs and substantially reduces cost. The key requirements are that the index test result and reference standard outcome are recorded with sufficient completeness and accuracy, that the time window between screening and reference-standard ascertainment is clinically appropriate, and that selective recording of reference-standard results (typically only among screen-positives or symptomatic patients) is identified and addressed.

Sample size is driven by the target sensitivity or specificity estimate, desired confidence interval width, and the expected prevalence of the condition in the study population. Low-prevalence conditions require very large samples to obtain precise sensitivity estimates because few cases are detected. Standard formulas are based on the expected number of true positives and true negatives; diagnostic accuracy calculators in software such as R (pROC package) or online STARD-based tools implement this. Always power for subgroup analyses if between-site heterogeneity is a primary research question.