Online Controlled Experiment
Online controlled experiments, commonly called A/B tests, randomly split live web or app traffic between a control and one or more treatment variants to measure the causal effect of a change on user behavior. Ron Kohavi, Diane Tang, and Ya Xu — who built and ran experimentation platforms at Microsoft, Google, and LinkedIn — set out the modern theory and best practice in their 2020 Cambridge book, and Kohavi's earlier survey with colleagues established the practical foundations of running trustworthy web experiments at scale. The discipline centers on a clearly defined Overall Evaluation Criterion (OEC) that captures long-term value, rigorous randomization, adequate statistical power, and a battery of trustworthiness checks such as the Sample Ratio Mismatch test. Because users are randomized, the difference in metrics between variants is an unbiased estimate of the change's causal impact — the gold standard for marketing and product decisions that attribution and observational analysis can only approximate. The output is a confident ship/no-ship decision: did this headline, layout, price, or feature actually move the metrics that matter, by how much, and with what certainty?
Catatan sumber
Kutipan disalin apa adanya dari catatan sumber metode. Tidak ada verifikasi tingkat klaim yang disimpulkan darinya.
- Kohavi, R., Tang, D., & Xu, Y. (2020). Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press. · ISBN 9781108724265
- Kohavi, R., Longbotham, R., Sommerfield, D., & Henne, R. M. (2009). Controlled experiments on the web: survey and practical guide. Data Mining and Knowledge Discovery, 18(1), 140-181. · DOI 10.1007/s10618-008-0114-1
Klaim yang dikurasi
Klaim tersimpan dalam buku besar bukti, masing-masing dengan penilaiannya sendiri.
Tampilan ini tidak menciptakan penilaian klaim ketika buku besar tidak memilikinya.
Metode terkait
Dihasilkan dari grafik metode dan ditampilkan sebagai relasi yang disarankan mesin — tidak ada klaim bukti yang disimpulkan.