N-gram Analysis
Also known as: Lexical Bundle Analysis, Cluster Analysis (corpus linguistics), Contiguous Sequence Analysis
N-gram analysis is a corpus-linguistic technique that extracts and ranks every contiguous sequence of n words (or characters) in a corpus, exposing the recurrent multi-word units — two-word bigrams, three-word trigrams, and longer 'lexical bundles' — that make up a register or text type. By counting how often each sequence recurs, it reveals the prefabricated, formulaic backbone of language that single-word frequency lists cannot capture.
Key highlights
- Captures the formulaic, prefabricated multi-word units that single-word frequency lists miss entirely.
- Fully automatic and language-agnostic, requiring no syntactic annotation or theoretical assumptions.
- Produces ranked, normalized counts that support rigorous quantitative comparison across registers and corpora.
- Scales effortlessly to billions of words and underlies practical applications from language modeling to plagiarism detection.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use n-gram analysis when you want to characterize the recurrent, formulaic multi-word patterning of a register, genre, or author, or to compare such patterning across corpora — for example, contrasting the lexical bundles of student and expert academic writing. It underpins lexical-bundle research, phraseology, language testing, and statistical language modeling. It is less suited to capturing discontinuous or syntactically flexible patterns (where collocation or collostructional methods are better), and very long n yields sparse counts that need large corpora to be meaningful.
Strengths & limitations
- Captures the formulaic, prefabricated multi-word units that single-word frequency lists miss entirely.
- Fully automatic and language-agnostic, requiring no syntactic annotation or theoretical assumptions.
- Produces ranked, normalized counts that support rigorous quantitative comparison across registers and corpora.
- Scales effortlessly to billions of words and underlies practical applications from language modeling to plagiarism detection.
- Only contiguous sequences are captured, so discontinuous patterns ('take ... into account') are split and missed.
- Counts are dominated by frequent function-word sequences unless frequency and dispersion filters are applied.
- Longer n-grams become sparse quickly, demanding very large corpora for stable estimates.
- Frequency alone does not establish that a sequence is psychologically a 'unit'; functional interpretation remains a separate, partly subjective step.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What is the difference between n-gram analysis and collocation analysis?
N-gram analysis counts contiguous sequences of fixed length and ranks them by raw or normalized frequency, capturing formulaic chunks regardless of statistical attraction. Collocation analysis measures the statistical association between a node word and words in its window (using measures such as mutual information or log-likelihood), can span gaps, and asks whether co-occurrence exceeds chance. N-grams answer 'what recurs', collocation answers 'what is attracted'.
What are lexical bundles?
Lexical bundles are the most frequent recurrent contiguous word sequences (typically three to six words) in a register, identified by frequency and dispersion thresholds. Examples include 'on the other hand' and 'as a result of'. Biber's work classifies them functionally into stance bundles, discourse organizers, and referential bundles, and shows that registers have distinctive bundle profiles.
How do I choose the value of n?
The choice depends on the question and the corpus size. Bigrams and trigrams capture tight collocational chunks and remain frequent enough for small corpora; four- to six-grams capture the lexical bundles studied in register research but require large corpora to occur often enough. Larger n always yields sparser counts, so n is a trade-off between pattern length and statistical reliability.
Sources
- 1.Biber, D., Johansson, S., Leech, G., Conrad, S., & Finegan, E. (1999). Longman Grammar of Spoken and Written English. Longman.ISBN 9780582237254
- 2.O'Keeffe, A., & McCarthy, M. (Eds.). (2010). The Routledge Handbook of Corpus Linguistics. Routledge.ISBN 9780415464895
- 3.Anthony, L. (2004). AntConc: A learner and classroom friendly, multi-platform corpus analysis toolkit. In Proceedings of IWLeL 2004: An Interactive Workshop on Language e-Learning (pp. 7–13). Waseda University.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 22). N-gram Analysis. ScholarGate. https://scholargate.app/linguistics/n-gram-analysis