Part-of-Speech Tagging
Also known as: POS Tagging, Grammatical Tagging, Word-Category Disambiguation
Part-of-speech (POS) tagging is the task of assigning each word (token) in a text its grammatical category — noun, verb, adjective, preposition, and finer distinctions such as past-tense verb or comparative adjective — drawn from a fixed tagset. Because the same word form can belong to different categories depending on context ("book a flight" versus "read a book"), tagging is fundamentally a disambiguation problem solved with contextual evidence. It is one of the oldest and most foundational tasks in natural language processing and corpus linguistics, supplying the grammatical layer on which concordancing, parsing, register analysis, and information extraction all depend. Modern taggers reach accuracies well above 97% on standard English benchmarks, using statistical sequence models or neural networks trained on annotated corpora.
Key highlights
- Provides a foundational grammatical layer that almost every other corpus and NLP task can build on.
- Achieves very high accuracy (often above 97% on standard English) with mature, fast, well-understood models.
- Resolves lexical category ambiguity using context, far beyond what dictionary lookup alone can do.
- Standard tagsets such as Penn Treebank and CLAWS make annotations comparable and reusable across projects and tools.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use POS tagging whenever a downstream task needs to know the grammatical category of words: building part-of-speech-annotated corpora, running tag-based concordance or collocation queries ("all instances of run as a noun"), counting grammatical features for register or stylometric analysis, feeding a syntactic parser, or filtering candidate terms in information extraction. It is essential preprocessing for most corpus-linguistic and NLP pipelines on English and many other languages. It is less helpful, or needs careful adaptation, for languages with rich morphology or no clear word boundaries, for very noisy social-media text, and for tasks where fine grammatical distinctions add error without analytic value.
Strengths & limitations
- Provides a foundational grammatical layer that almost every other corpus and NLP task can build on.
- Achieves very high accuracy (often above 97% on standard English) with mature, fast, well-understood models.
- Resolves lexical category ambiguity using context, far beyond what dictionary lookup alone can do.
- Standard tagsets such as Penn Treebank and CLAWS make annotations comparable and reusable across projects and tools.
- Accuracy drops sharply on text unlike the training data — historical, learner, dialectal, or social-media language.
- Errors concentrate on genuinely ambiguous words and on unknown words, and these errors propagate to downstream tasks.
- Tagset granularity is a fixed trade-off: finer tags capture more grammar but introduce more disambiguation errors.
- Morphologically rich and segmentation-free languages need adapted tagsets and tokenizers, reducing the portability of English-centric methods.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
Why is POS tagging not just dictionary lookup?
Many word forms belong to more than one category — "book", "run", "that", and "like" can each be several parts of speech. A dictionary lists the possibilities, but only context tells you which one applies in a given sentence. POS tagging is precisely the task of choosing the correct category from those possibilities using the surrounding words and tags, which is why it is modeled as sequence disambiguation rather than a lookup.
What is the Penn Treebank tagset?
The Penn Treebank tagset is a set of about 36 part-of-speech labels (plus punctuation tags) introduced with the Penn Treebank in 1993. It makes fine grammatical distinctions — for instance separating singular nouns (NN) from plural (NNS), and base, past, and gerund verb forms (VB, VBD, VBG). Its wide adoption as training and evaluation data made it the standard tagset for English NLP, though corpora such as the BNC use the alternative CLAWS tagsets.
How accurate are modern POS taggers?
On standard English benchmarks like the Penn Treebank, well-trained statistical and neural taggers exceed 97% token accuracy. However, that figure is inflated by the many words that are never ambiguous; accuracy on genuinely ambiguous tokens and on unknown words is lower, and overall accuracy falls considerably on text that differs from the training data, such as historical, learner, or social-media language.
Sources
- 1.Manning, C. D., Raghavan, P., & Schütze, H. (2008). Introduction to Information Retrieval. Cambridge University Press.ISBN 9780521865715
- 2.Jurafsky, D., & Martin, J. H. (2023). Speech and Language Processing (3rd ed. draft). Stanford University.
- 3.Marcus, M. P., Marcinkiewicz, M. A., & Santorini, B. (1993). Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2), 313–330.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 22). Part-of-Speech Tagging. ScholarGate. https://scholargate.app/linguistics/part-of-speech-tagging