Domain-adaptive Text Summarization
Domain-adaptive Text Summarization (Domain Adaptation for Abstractive and Extractive Summarization) · Also known as: domain-adapted summarization, domain-specific summarization, cross-domain summarization, DA-summarization
Domain-adaptive text summarization fine-tunes or adapts a pre-trained sequence-to-sequence language model on a target domain corpus so that summaries conform to domain-specific vocabulary, style, and factual constraints. It bridges the gap between general-purpose summarization models trained on news or web data and specialized domains such as biomedical literature, legal documents, scientific papers, or financial reports.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use domain-adaptive text summarization when you need to automatically condense domain-specific documents (biomedical, legal, financial, scientific) and a general-purpose summarizer produces inaccurate terminology, wrong length, or hallucinated facts. It is appropriate when at least a few hundred in-domain document-summary pairs are available for fine-tuning, or when a large unlabelled in-domain corpus can support DAPT. Do not use it when the target domain has virtually no training data and the general model already performs acceptably; a zero-shot approach may suffice. Avoid it when factual precision is mission-critical and no faithfulness evaluation pipeline is in place, as all neural summarizers can hallucinate.
Strengths & limitations
- Significantly improves ROUGE and factual faithfulness compared with zero-shot general models on domain text.
- Inherits strong language generation from large pre-trained models, reducing the data needed for adaptation.
- Flexible: adapts extractive, abstractive, or hybrid strategies to the target domain.
- Domain-adaptive pre-training on unlabelled text extends coverage to rare in-domain terms without requiring labelled pairs.
- Supports multi-document and long-document settings through sliding-window or hierarchical encoder extensions.
- Requires labelled in-domain document-summary pairs; data collection is expensive in specialized domains.
- Neural abstractive models can hallucinate facts, which is particularly dangerous in medical or legal contexts.
- Long documents exceed typical transformer context windows, requiring truncation or hierarchical processing.
- Evaluation is primarily automatic (ROUGE); ROUGE correlates weakly with human judgment for domain-specific quality.
- Computational cost of fine-tuning large models (BART-large, T5-large) may be prohibitive without GPU resources.
Frequently asked
How many training examples are needed for effective domain adaptation?
With DAPT on unlabelled text followed by fine-tuning, a few hundred to a few thousand labelled pairs often suffice. Without DAPT, at least 1,000–5,000 pairs are advisable to avoid catastrophic forgetting. In very low-resource settings, data augmentation or cross-domain transfer from a related domain can help.
How do I reduce hallucinations in domain-adaptive summarization?
Apply factual consistency post-processing filters (e.g., QAFactEval, FactCC), constrain generation with entity-copy mechanisms, and always conduct domain-expert evaluation before deployment in high-stakes settings.
Should I use BART, T5, or PEGASUS as the base model?
PEGASUS is designed for summarization and often adapts fastest with limited data. BART-large is a strong general-purpose choice. T5 is flexible and scales well. Choose based on available GPU memory and whether the domain is more abstractive (PEGASUS/BART) or requires controllable generation (T5 with task prefixes).
Is ROUGE a reliable evaluation metric for domain-specific summaries?
ROUGE measures n-gram overlap with reference summaries and often fails to capture domain-specific quality such as factual correctness or terminology fidelity. Complement it with BERTScore, faithfulness metrics, and human evaluation for any serious deployment.
Can domain-adaptive summarization work with very long documents?
Standard transformers are limited to around 512–1024 tokens. For long documents, use hierarchical encoders, sliding-window attention (Longformer), or chunk-then-merge strategies. PEGASUS and LED (Longformer Encoder-Decoder) are specifically designed to handle longer inputs.
Sources
- Fabbri, A. R., KryŜiński, W., McCann, B., Xiong, C., Socher, R., & Radev, D. (2021). SummEval: Re-evaluating Summarization Evaluation. Transactions of the Association for Computational Linguistics, 9, 391–409. DOI: 10.1162/tacl_a_00373 ↗
- Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020). On Faithfulness and Factuality in Abstractive Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020), pp. 1906–1919. DOI: 10.18653/v1/2020.acl-main.173 ↗
How to cite this page
ScholarGate. (2026, June 3). Domain-adaptive Text Summarization (Domain Adaptation for Abstractive and Extractive Summarization). ScholarGate. https://scholargate.app/en/deep-learning/domain-adaptive-text-summarization
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- BERT-based ClassificationDeep learning↔ compare
- Domain-adaptive BERT-based ClassificationDeep learning↔ compare
- Domain-adaptive Named Entity RecognitionDeep learning↔ compare
- Fine-Tuned Text SummarizationDeep learning↔ compare
- Multimodal Text SummarizationDeep learning↔ compare
- Transfer Learning with Text SummarizationDeep learning↔ compare