השוואת שיטות

	Cranfield Evaluation Paradigm	Relevance Feedback Evaluation
תחום	Library Information Science	Library Information Science
משפחה	Process / pipeline	Process / pipeline
שנת המקור	1967	1990
הוגה השיטה	Cyril W. Cleverdon	Gerard Salton & Chris Buckley (building on J. J. Rocchio)
סוג	Test-collection evaluation pipeline for retrieval effectiveness	Evaluation pipeline for relevance-feedback query reformulation
מקור מכונן	Cleverdon, C. W. (1967). The Cranfield tests on index language devices. Aslib Proceedings, 19(6), 173-194.	Salton, G., & Buckley, C. (1990). Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science, 41(4), 288-297.
כינויים	Cranfield Methodology, Test Collection Evaluation, Cranfield Tests, Laboratory IR Evaluation	Rocchio Feedback Evaluation, Feedback Effectiveness Measurement, Residual Collection Evaluation, Relevance Feedback Assessment
קשורות	3	3
תקציר	The Cranfield evaluation paradigm is the foundational experimental design for measuring how well an information retrieval system finds relevant documents. Devised by Cyril Cleverdon at the College of Aeronautics in Cranfield during the 1960s, it fixes three ingredients — a document collection, a set of search requests, and human relevance judgments linking requests to documents — and then holds them constant so that competing indexing methods or retrieval algorithms can be compared on recall and precision under controlled, repeatable conditions. By abstracting evaluation away from any single live user and turning it into a reusable laboratory experiment, Cranfield made retrieval effectiveness a measurable quantity and supplied the template that every later large-scale campaign, including TREC, has built upon.	Relevance feedback evaluation measures how much a retrieval system improves when it reformulates a query using user judgments on the first results. The technique that defined the field is Rocchio's vector-space feedback, in which documents the user marks relevant pull the query vector toward themselves and documents marked non-relevant push it away; Salton and Buckley's 1990 study systematized its evaluation and showed substantial effectiveness gains. The central methodological challenge is fairness: because the user has already seen and judged some documents, naively re-scoring the whole collection rewards the system for re-finding documents it was just told about. Residual-collection and frozen-rank evaluation solve this by measuring improvement only on documents the user has not yet seen.