Text Infilling — Cloze Completion
Text Infilling (Cloze Completion) · Also known as: cloze procedure, cloze test, masked language modeling, span infilling, Metin Doldurma (Text Infilling / Cloze)
Text infilling is a natural-language-processing task that completes missing words, phrases, or spans in a document by exploiting the surrounding context. Introduced as the cloze procedure by Wilson L. Taylor in 1953 as a readability measure, it was reformulated for neural models by Zhu et al. (2019) and is now used for data augmentation, writing assistance, and language-model evaluation.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Text infilling is appropriate when you have text data with explicit gaps or when you need to synthesise plausible continuations inside an existing passage. It is well suited to data augmentation (generating varied training examples), automated writing assistance (suggesting missing content), cloze-style readability tests, and diagnosing language model quality. The method requires a pretrained language model; a large model is preferred for span-level tasks. The source text must define mask or blank positions clearly, and at least ten text examples should be available to produce meaningful evaluation results.
Strengths & limitations
- Leverages bidirectional context — unlike left-to-right generation, the model sees both sides of a gap and produces more coherent fills.
- Flexible task scope: a single masked token up to entire missing paragraphs can be predicted within the same framework.
- Dual utility as both a downstream application (writing assistance, data augmentation) and a diagnostic probe for evaluating language model quality.
- A large pretrained language model is preferred; without one, prediction quality degrades substantially.
- Span-level infilling can produce fluent but factually incorrect completions, especially when world knowledge is required.
- The method requires mask positions to be defined in advance; it cannot discover what is missing from a text autonomously.
Frequently asked
What is the difference between text infilling and text generation?
Text generation predicts what comes after a prefix (left-to-right). Text infilling predicts content that fits inside an existing passage, using both the left and right surrounding context. This bilateral constraint makes infilling harder but produces more coherent results when the surrounding text is fixed.
Can I use BERT for span infilling?
BERT predicts one masked token at a time and is designed for single-token cloze tasks. For filling spans of multiple tokens, use a sequence-to-sequence model such as T5 or BART, which can generate variable-length outputs conditioned on the full passage.
How do I evaluate the quality of infilled text?
For single-token fills, accuracy at rank-1 against a reference token is the standard metric. For spans, BLEU and ROUGE compare the predicted fill to a reference, while perplexity measures fluency under a separate language model. Because multiple valid fills often exist, human fluency and coherence judgments are a valuable complement to automated scores.
Does text infilling require labelled data?
For application to new text, no labelled data is needed — only a pretrained language model and a source text with defined mask positions. Labelled data (texts with known correct fills) is needed only for evaluation or for fine-tuning a model on a specific domain.
Sources
How to cite this page
ScholarGate. (2026, June 1). Text Infilling (Cloze Completion). ScholarGate. https://scholargate.app/en/text-mining/text-infilling
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- BERT EmbeddingsText mining↔ compare
- Named Entity RecognitionText mining↔ compare
- Sentiment AnalysisText mining↔ compare
- Text ClassificationText mining↔ compare