Сравнение методов

	Inter-Indexer Consistency	Cranfield Evaluation Paradigm
Область	Library Information Science	Library Information Science
Семейство	Process / pipeline	Process / pipeline
Год появления	1969	1967
Автор метода	Pranas Zunde & Margaret Dexter; L. Rolling (Hooper coefficient tradition)	Cyril W. Cleverdon
Тип	Agreement measurement for index-term assignment	Test-collection evaluation pipeline for retrieval effectiveness
Основополагающий источник	Rolling, L. (1981). Indexing consistency, quality and efficiency. Information Processing & Management, 17(2), 69-76.	Cleverdon, C. W. (1967). The Cranfield tests on index language devices. Aslib Proceedings, 19(6), 173-194.
Другие названия	Indexer Agreement, Inter-Indexer Agreement, Hooper Consistency, Indexing Reliability	Cranfield Methodology, Test Collection Evaluation, Cranfield Tests, Laboratory IR Evaluation
Связанные	3	3
Сводка	Inter-indexer consistency measures how far two or more people agree when they independently assign subject terms to the same documents. Because subject indexing is a judgment task — choosing which descriptors best represent a document's content — different indexers routinely pick overlapping but not identical term sets, and the degree of that overlap is a fundamental indicator of the reliability of an indexing system. The standard quantity is the Hooper-style consistency coefficient, the size of the shared term set divided by the size of the combined term set, averaged across documents; Zunde and Dexter and later Rolling refined it and connected it to indexing quality. Low consistency signals that retrieval will be unpredictable, since whether a document is found can depend on which indexer happened to process it.	The Cranfield evaluation paradigm is the foundational experimental design for measuring how well an information retrieval system finds relevant documents. Devised by Cyril Cleverdon at the College of Aeronautics in Cranfield during the 1960s, it fixes three ingredients — a document collection, a set of search requests, and human relevance judgments linking requests to documents — and then holds them constant so that competing indexing methods or retrieval algorithms can be compared on recall and precision under controlled, repeatable conditions. By abstracting evaluation away from any single live user and turning it into a reusable laboratory experiment, Cranfield made retrieval effectiveness a measurable quantity and supplied the template that every later large-scale campaign, including TREC, has built upon.