Process / pipelineExperimental design
Pragmatic A/B Test — Real-World Randomized Comparison
A pragmatic A/B test is a randomized comparative experiment that evaluates two alternatives — a control (A) and a treatment (B) — under real-world operating conditions rather than tightly controlled laboratory settings. Rooted in the pragmatic-versus-explanatory trial distinction introduced by Schwartz and Lellouch in 1967 and brought to large-scale practice by online experimentation teams at Microsoft, Google, and Amazon, it prioritizes decision-relevant effectiveness over internal mechanistic explanation.
Find Topic with PaperMindSoonVideoSoon
Read the full method
Members only
Sign inSign in with a free account to read this section.
Sources
- Schwartz, D., & Lellouch, J. (1967). Explanatory and pragmatic attitudes in therapeutical trials. Journal of Chronic Diseases, 20(8), 637–648. DOI: 10.1016/0021-9681(67)90041-0 ↗
- Kohavi, R., Tang, D., & Xu, Y. (2020). Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press. ISBN: 978-1108724265