Sammenlign metoder
Gennemgå dine valgte metoder side om side; rækker, der afviger, er fremhævet.
| Cranfield Evaluation Paradigm× | TREC Pooling and Relevance Judgments× | |
|---|---|---|
| Fagområde | Library Information Science | Library Information Science |
| Familie | Process / pipeline | Process / pipeline |
| Oprindelsesår≠ | 1967 | 2005 |
| Ophavsperson≠ | Cyril W. Cleverdon | Ellen M. Voorhees & Donna K. Harman (NIST TREC) |
| Type≠ | Test-collection evaluation pipeline for retrieval effectiveness | Pooled relevance-assessment pipeline for large test collections |
| Oprindelig kilde≠ | Cleverdon, C. W. (1967). The Cranfield tests on index language devices. Aslib Proceedings, 19(6), 173-194. DOI ↗ | Voorhees, E. M., & Harman, D. K. (Eds.). (2005). TREC: Experiment and Evaluation in Information Retrieval. MIT Press. ISBN: 9780262220736 |
| Aliasser | Cranfield Methodology, Test Collection Evaluation, Cranfield Tests, Laboratory IR Evaluation | Pooling Method, Depth Pooling, TREC Pooling, Pooled Relevance Assessment |
| Relaterede | 3 | 3 |
| Resumé≠ | The Cranfield evaluation paradigm is the foundational experimental design for measuring how well an information retrieval system finds relevant documents. Devised by Cyril Cleverdon at the College of Aeronautics in Cranfield during the 1960s, it fixes three ingredients — a document collection, a set of search requests, and human relevance judgments linking requests to documents — and then holds them constant so that competing indexing methods or retrieval algorithms can be compared on recall and precision under controlled, repeatable conditions. By abstracting evaluation away from any single live user and turning it into a reusable laboratory experiment, Cranfield made retrieval effectiveness a measurable quantity and supplied the template that every later large-scale campaign, including TREC, has built upon. | Pooling is the technique that lets the Cranfield evaluation paradigm scale to collections of millions of documents, where judging every document for every topic is impossible. Developed and institutionalized at the US National Institute of Standards and Technology for the Text REtrieval Conference (TREC), pooling gathers the top-ranked documents returned by many participating systems for each topic, merges them into a single pool, has human assessors judge only that pool, and treats every unjudged document as non-relevant. The result is a reusable test collection — documents, topics, and pooled relevance judgments (qrels) — on which new systems can later be scored without further assessment. Pooling is what made large-scale, reproducible retrieval evaluation feasible. |
| ScholarGateDatasæt ↗ |
|
|