Weakly Supervised Text Summarization
Also known as: weak supervision summarization, distantly supervised summarization, noisy-label summarization, pseudo-label summarization
Weakly supervised text summarization trains abstractive or extractive summarization models without manually annotated reference summaries. Instead of costly human labels, it exploits weak signals — heuristic rules, distant supervision, noisy automatic labels, or self-supervised objectives — to guide sequence-to-sequence or transformer models toward producing coherent, concise summaries of input documents.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use weakly supervised summarization when manually annotated (document, summary) pairs are scarce, expensive, or unavailable for the target domain, language, or document type. It is well suited to low-resource settings such as scientific literature, legal documents, medical records, or domain-specific corpora where labeled data collection is prohibitive. Not recommended when high factual accuracy is critical and noise in pseudo-labels cannot be adequately filtered, or when a sufficiently large labeled dataset is already available, in which case fully supervised fine-tuning of a pre-trained model (BART, T5, PEGASUS) is preferred.
Strengths & limitations
- Eliminates or drastically reduces the need for expensive human annotation of summary pairs.
- Scales naturally to low-resource languages and specialized domains where labeled data is scarce.
- Compatible with large pre-trained language models (BART, T5, PEGASUS) whose representations reduce label-noise sensitivity.
- Flexible: diverse weak signals (heuristics, distant supervision, self-supervision) can be combined.
- Enables domain adaptation without full re-annotation when a new document type is encountered.
- Pseudo-label noise can propagate into the model, producing unfaithful or hallucinated summaries.
- Evaluation is difficult because automatic metrics like ROUGE do not capture factuality or coherence well.
- Weaker signals may not reflect the diversity or style of ideal human-written summaries.
- Noise-handling strategies (curriculum learning, confidence weighting) add training complexity.
- Generalization to out-of-domain documents can be limited if the weak-signal source is domain-specific.
Frequently asked
What is the difference between weakly supervised and unsupervised summarization?
Weakly supervised methods use some indirect or noisy training signal (pseudo-labels, heuristics, distant supervision) to guide the model, whereas unsupervised methods use no target labels at all, relying purely on extractive criteria or reconstruction objectives. Weak supervision sits between fully supervised and fully unsupervised on the supervision spectrum.
Is ROUGE a reliable evaluation metric for weakly supervised summarizers?
ROUGE measures n-gram overlap with reference summaries and is the standard automatic benchmark, but it does not capture factual faithfulness or coherence. For weakly supervised models, which are prone to hallucination, supplementing ROUGE with faithfulness metrics (FactCC, DAE) or human evaluation is strongly recommended.
Can I use a pre-trained model like BART or T5 with weak supervision?
Yes, and this is the dominant approach. Pre-trained models already encode strong language priors, so fine-tuning on noisy pseudo-labels is less damaging than training from scratch. PEGASUS in particular was pre-trained with a gap-sentence generation objective that mimics summarization, making it especially effective in low-label regimes.
What weak signals work best for scientific text summarization?
Author-written abstracts paired with their full papers are the most natural weak signal for scientific summarization. Section headings, citation contexts, and structured metadata (keywords, highlights) have also been used when abstract-paper pairs are unavailable in the target domain.
How do I handle the lead bias introduced by using lead sentences as pseudo-labels?
Lead bias occurs when models learn to copy opening sentences because pseudo-labels favor them. Mitigation strategies include using ROUGE-optimized sentence selection rather than positional heuristics, applying sentence-position penalties in training objectives, or augmenting pseudo-labels with non-leading salient sentences identified by TF-IDF or graph-based ranking.
Sources
- Amplayo, R. K., & Lapata, M. (2020). Unsupervised Opinion Summarization with Noisy Autoencoder. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 1934–1945. link ↗
- Huang, L., Wu, L., & Wang, L. (2020). Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5094–5107. link ↗
How to cite this page
ScholarGate. (2026, June 3). Weakly Supervised Text Summarization. ScholarGate. https://scholargate.app/en/deep-learning/weakly-supervised-text-summarization
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Self-supervised LearningMachine learning↔ compare