TREC Pooling and Relevance Judgments
Pooling is the technique that lets the Cranfield evaluation paradigm scale to collections of millions of documents, where judging every document for every topic is impossible. Developed and institutionalized at the US National Institute of Standards and Technology for the Text REtrieval Conference (TREC), pooling gathers the top-ranked documents returned by many participating systems for each topic, merges them into a single pool, has human assessors judge only that pool, and treats every unjudged document as non-relevant. The result is a reusable test collection — documents, topics, and pooled relevance judgments (qrels) — on which new systems can later be scored without further assessment. Pooling is what made large-scale, reproducible retrieval evaluation feasible.
Catatan sumber
Kutipan disalin apa adanya dari catatan sumber metode. Tidak ada verifikasi tingkat klaim yang disimpulkan darinya.
- Voorhees, E. M., & Harman, D. K. (Eds.). (2005). TREC: Experiment and Evaluation in Information Retrieval. MIT Press. · ISBN 9780262220736
- Manning, C. D., Raghavan, P., & Schütze, H. (2008). Introduction to Information Retrieval. Cambridge University Press. · ISBN 9780521865715
- Cleverdon, C. W. (1967). The Cranfield tests on index language devices. Aslib Proceedings, 19(6), 173-194. · DOI 10.1108/eb050097
Klaim yang dikurasi
Klaim tersimpan dalam buku besar bukti, masing-masing dengan penilaiannya sendiri.
Tampilan ini tidak menciptakan penilaian klaim ketika buku besar tidak memilikinya.
Metode terkait
Dihasilkan dari grafik metode dan ditampilkan sebagai relasi yang disarankan mesin — tidak ada klaim bukti yang disimpulkan.