Mean Average Precision (MAP)
Also known as: MAP, Average Precision, AP, Mean AP
Mean Average Precision (MAP) is the classic single-number summary of ranked-retrieval effectiveness under binary relevance and the headline metric of the TREC ad hoc retrieval tracks. For a single query, average precision (AP) computes the precision of the result list at each rank where a relevant document appears and averages those values, rewarding systems that rank all relevant documents highly; MAP is then the mean of AP across a set of queries. Buckley and Voorhees's 2000 SIGIR analysis of evaluation-measure stability showed that average precision is among the most stable and discriminating IR measures, requiring fewer queries than alternatives like precision at a fixed cutoff to reliably tell two systems apart. MAP remains a standard reporting metric for ranked retrieval, complementing graded-relevance measures such as nDCG.
Key highlights
- Single, interpretable scalar that summarizes ranking quality across the whole result list, not just a fixed cutoff.
- Recall-aware: rewards retrieving and highly ranking all relevant documents, penalizing systems that miss relevant items.
- Empirically stable and discriminating, requiring relatively few queries to reliably distinguish systems (Buckley and Voorhees).
- Long-standing TREC standard, enabling direct comparison with a large body of published retrieval results.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use MAP when you evaluate ranked retrieval under binary relevance and want a single, stable, recall-aware score that rewards ranking all relevant documents highly: TREC-style ad hoc retrieval, system comparisons in IR research, and any task where the complete set of relevant documents matters rather than just the top few. It is especially appropriate when you have a sufficiently large query set and known relevant-document sets, because Buckley and Voorhees showed average precision discriminates systems reliably with fewer queries than alternatives. MAP is less suitable when relevance is graded rather than binary (use nDCG), when only top-of-list quality matters and recall is irrelevant (precision at k or reciprocal rank may fit better), or when relevance judgments are badly incomplete, since unjudged-as-non-relevant assumptions bias the recall normalization.
Strengths & limitations
- Single, interpretable scalar that summarizes ranking quality across the whole result list, not just a fixed cutoff.
- Recall-aware: rewards retrieving and highly ranking all relevant documents, penalizing systems that miss relevant items.
- Empirically stable and discriminating, requiring relatively few queries to reliably distinguish systems (Buckley and Voorhees).
- Long-standing TREC standard, enabling direct comparison with a large body of published retrieval results.
- Assumes binary relevance and cannot express degrees of relevance, unlike nDCG.
- Requires the full set of relevant documents to be known; with incomplete judgments it is biased by treating unjudged documents as non-relevant.
- Weights all ranks through the whole list, so it may not reflect tasks where only the very top results matter to users.
- Averaging AP equally across queries can let a few easy or hard queries dominate the summary unless query sets are well designed.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How is average precision different from precision at k?
Precision at k is the fraction of relevant documents in the top k and ignores both their order within the cutoff and any relevant documents below it. Average precision instead samples precision at every rank where a relevant document appears and averages over all relevant documents, so it is recall-aware and rewards ranking all relevant items highly. MAP is the mean of average precision across queries, giving a fuller picture of ranking quality than precision at a single cutoff.
How many queries do I need for a reliable MAP comparison?
Buckley and Voorhees's stability analysis found that average precision discriminates systems with a relatively low error rate, and they recommend at least 25 queries, with 50 being substantially better. With too few queries the difference between two systems' MAP can be dominated by sampling noise, so reporting the query count and a significance test (such as a paired t-test or randomization test) is good practice.
When should I prefer nDCG over MAP?
Prefer nDCG when relevance is graded — when a highly relevant document should count more than a marginally relevant one — and when top-of-list quality at a cutoff is the priority, as in web search. Prefer MAP when relevance is binary and you care about ranking the entire set of relevant documents highly, as in recall-sensitive tasks. Many papers report both to characterize a system from both the binary, full-list view and the graded, top-heavy view.
Sources
- 1.Buckley, C., & Voorhees, E. M. (2000). Evaluating evaluation measure stability. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '00), 33-40.
- 2.Manning, C. D., Raghavan, P., & Schütze, H. (2008). Introduction to Information Retrieval. Cambridge University Press.ISBN 9780521865715
You have read it. What now?
Cite this page
ScholarGate. (2026, June 23). Mean Average Precision (MAP). ScholarGate. https://scholargate.app/bibliometrics/mean-average-precision