Compare methods

	Indexing Consistency Analysis	Query Log Analysis
Field	Library Information Science	Library Information Science
Family	Process / pipeline	Process / pipeline
Year of origin	1981	2000
Originator	L. Rolling; Pranas Zunde & Margaret Dexter (indexing-evaluation tradition)	Bernard J. Jansen, Amanda Spink & Tefko Saracevic; Silverstein et al.
Type	Diagnostic analysis of indexing variability and its retrieval consequences	Transaction-log analysis pipeline for search behavior
Seminal source	Rolling, L. (1981). Indexing consistency, quality and efficiency. Information Processing & Management, 17(2), 69-76.	Jansen, B. J., Spink, A., & Saracevic, T. (2000). Real life, real users, and real needs: a study and analysis of user queries on the web. Information Processing & Management, 36(2), 207-227.
Aliases	Indexing Variability Analysis, Indexing Reliability Analysis, Indexing Consistency Study, Within- and Between-Indexer Consistency Analysis	Transaction Log Analysis, Search Log Analysis, Web Query Log Analysis, Query Transaction Analysis
Related	3	3
Summary	Indexing consistency analysis goes beyond reporting a single agreement number to diagnose why indexing varies and what that variability costs. It distinguishes between-indexer consistency (do different people agree?) from within-indexer consistency (does the same person agree with themselves on re-indexing?), models how factors such as indexing exhaustivity, vocabulary specificity, document subject, and indexer experience drive the variability, and — following Rolling's question of whether consistency stands in for quality — traces how inconsistency degrades retrieval. The aim is actionable: identify the terms, subjects, and conditions where indexers diverge most, and feed that back into guidelines, vocabulary design, and training.	Query log analysis — also called transaction-log analysis — studies the records that search systems automatically keep of what users typed, what they clicked, and when. Rather than asking users what they do or testing systems in the laboratory, it observes millions of real searches as they actually happened. The landmark studies by Jansen, Spink, and Saracevic on the Excite engine and by Silverstein and colleagues on AltaVista revealed a consistent and surprising picture: real web queries are very short, rarely use advanced operators, and users almost never look past the first page of results. By cleaning logs, reconstructing sessions, and tabulating term, query, and session statistics, the method turns raw server records into a behavioral portrait of how people really search.