השוואת שיטות

	Inter-Indexer Consistency	Cranfield Evaluation Paradigm
תחום	Library Information Science	Library Information Science
משפחה	Process / pipeline	Process / pipeline
שנת המקור	1969	1967
הוגה השיטה	Pranas Zunde & Margaret Dexter; L. Rolling (Hooper coefficient tradition)	Cyril W. Cleverdon
סוג	Agreement measurement for index-term assignment	Test-collection evaluation pipeline for retrieval effectiveness
מקור מכונן	Rolling, L. (1981). Indexing consistency, quality and efficiency. Information Processing & Management, 17(2), 69-76.	Cleverdon, C. W. (1967). The Cranfield tests on index language devices. Aslib Proceedings, 19(6), 173-194.
כינויים	Indexer Agreement, Inter-Indexer Agreement, Hooper Consistency, Indexing Reliability	Cranfield Methodology, Test Collection Evaluation, Cranfield Tests, Laboratory IR Evaluation
קשורות	3	3
תקציר	Inter-indexer consistency measures how far two or more people agree when they independently assign subject terms to the same documents. Because subject indexing is a judgment task — choosing which descriptors best represent a document's content — different indexers routinely pick overlapping but not identical term sets, and the degree of that overlap is a fundamental indicator of the reliability of an indexing system. The standard quantity is the Hooper-style consistency coefficient, the size of the shared term set divided by the size of the combined term set, averaged across documents; Zunde and Dexter and later Rolling refined it and connected it to indexing quality. Low consistency signals that retrieval will be unpredictable, since whether a document is found can depend on which indexer happened to process it.	The Cranfield evaluation paradigm is the foundational experimental design for measuring how well an information retrieval system finds relevant documents. Devised by Cyril Cleverdon at the College of Aeronautics in Cranfield during the 1960s, it fixes three ingredients — a document collection, a set of search requests, and human relevance judgments linking requests to documents — and then holds them constant so that competing indexing methods or retrieval algorithms can be compared on recall and precision under controlled, repeatable conditions. By abstracting evaluation away from any single live user and turning it into a reusable laboratory experiment, Cranfield made retrieval effectiveness a measurable quantity and supplied the template that every later large-scale campaign, including TREC, has built upon.