Text Coherence Scoring — Local Coherence Modeling
Text Coherence Scoring (Local Coherence Modeling) · Also known as: coherence modeling, local coherence assessment, Metin Tutarlılık Puanlaması
Text coherence scoring computes a document-level coherence score with machine learning, rooted in the entity-based local coherence model introduced by Barzilay and Lapata (2008). It measures how well the sentences of a text hang together, using either an entity-grid model, a graph-based approach, or a transformer-based model.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use text coherence scoring when you have text data and want to assess how well it reads as a connected whole — for automatic essay or text evaluation and for measuring summary quality. It needs at least a modest amount of text (around 20 units) and sentence order must be preserved. A reference text is not required, but having one makes the score more reliable.
Strengths & limitations
- Captures discourse-level structure that sentence-by-sentence methods miss, by tracking how entities reappear across a document.
- Flexible across approaches: an entity-grid model, a graph-based model, or a transformer can be used depending on resources.
- Reduces a whole document to a single, comparable coherence score suitable for automatic evaluation and summary-quality measurement.
- Requires that sentence order be preserved; shuffled or fragmented input invalidates the coherence signal.
- Scores are most reliable when a reference text is available; without one they are harder to calibrate.
- Needs a reasonable amount of text to produce a stable score, and entity-based models depend on correctly identified entities.
Frequently asked
What does a coherence score actually measure?
It measures how well the sentences of a text connect as a whole — whether entities are tracked and referred back to in a way that makes the discourse flow. It is a document-level signal about structure, not about grammar within a single sentence.
Entity-grid, graph-based, or transformer — which approach should I use?
The entity-grid model records each entity's grammatical role across sentences and is the classic approach. A graph-based model links sentences that share entities and summarises their connectivity. A transformer-based model encodes the sentence sequence directly and can capture more context, at higher computational cost.
Do I need a reference text?
No, but it helps. Coherence can be scored on a single document, yet the score is more reliable when compared against a reference text or human ratings, which is why the method records whether a reference is available.
Why does sentence order matter so much?
Local coherence is defined by how one sentence follows another and how entities carry over between them. If the original sentence order is not preserved, that transition signal is destroyed and the score no longer reflects the text's actual coherence.
Sources
- Barzilay, R. & Lapata, M. (2008). Modeling Local Coherence: An Entity-Based Approach. Computational Linguistics, 34(1), 1-34. DOI: 10.1162/coli.2008.34.1.1 ↗
- Guinaudeau, C. & Strube, M. (2013). Graph-based Local Coherence Modeling. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL), 93-103. link ↗
How to cite this page
ScholarGate. (2026, June 1). Text Coherence Scoring (Local Coherence Modeling). ScholarGate. https://scholargate.app/en/text-mining/text-coherence-scoring
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Automatic Text EvaluationText mining↔ compare
- BERT EmbeddingsText mining↔ compare
- Sentiment AnalysisText mining↔ compare
- Text ClassificationText mining↔ compare