Módszerek összehasonlítása

	Relevance Feedback Evaluation	Cranfield Evaluation Paradigm
Tudományterület	Library Information Science	Library Information Science
Módszercsalád	Process / pipeline	Process / pipeline
Keletkezés éve	1990	1967
Megalkotó	Gerard Salton & Chris Buckley (building on J. J. Rocchio)	Cyril W. Cleverdon
Típus	Evaluation pipeline for relevance-feedback query reformulation	Test-collection evaluation pipeline for retrieval effectiveness
Alapmű	Salton, G., & Buckley, C. (1990). Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science, 41(4), 288-297.	Cleverdon, C. W. (1967). The Cranfield tests on index language devices. Aslib Proceedings, 19(6), 173-194.
Alternatív nevek	Rocchio Feedback Evaluation, Feedback Effectiveness Measurement, Residual Collection Evaluation, Relevance Feedback Assessment	Cranfield Methodology, Test Collection Evaluation, Cranfield Tests, Laboratory IR Evaluation
Kapcsolódó	3	3
Összefoglaló	Relevance feedback evaluation measures how much a retrieval system improves when it reformulates a query using user judgments on the first results. The technique that defined the field is Rocchio's vector-space feedback, in which documents the user marks relevant pull the query vector toward themselves and documents marked non-relevant push it away; Salton and Buckley's 1990 study systematized its evaluation and showed substantial effectiveness gains. The central methodological challenge is fairness: because the user has already seen and judged some documents, naively re-scoring the whole collection rewards the system for re-finding documents it was just told about. Residual-collection and frozen-rank evaluation solve this by measuring improvement only on documents the user has not yet seen.	The Cranfield evaluation paradigm is the foundational experimental design for measuring how well an information retrieval system finds relevant documents. Devised by Cyril Cleverdon at the College of Aeronautics in Cranfield during the 1960s, it fixes three ingredients — a document collection, a set of search requests, and human relevance judgments linking requests to documents — and then holds them constant so that competing indexing methods or retrieval algorithms can be compared on recall and precision under controlled, repeatable conditions. By abstracting evaluation away from any single live user and turning it into a reusable laboratory experiment, Cranfield made retrieval effectiveness a measurable quantity and supplied the template that every later large-scale campaign, including TREC, has built upon.