手法を比較

	ドメイン適応型BERTベース分類	ドメイン適応型Transformer
分野	深層学習	深層学習
系統	Machine learning	Machine learning
提唱年	2019–2020	2019–2022
提唱者	Gururangan et al. (2020); earlier domain-specific instances include Lee et al. (2020) — BioBERT	Various (Vaswani et al. 2017 for Transformers; domain adaptation extensions emerged 2019–2022)
種類	Domain-adaptive pre-training followed by supervised fine-tuning	Pre-trained model fine-tuned with domain-shift adaptation
原典	Gururangan, S., Marasovic, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020). Don't Stop Pretraining: Adapt Language Models to Domains and Tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020), 8342–8360.	Ni, J., Hernandez Abrego, G., Constant, N., Ma, J., Hall, K., Cer, D., & Yang, Y. (2021). Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models. Findings of ACL 2022. arXiv:2108.08877.
別名	DAPT BERT classification, domain-adaptive pre-training, domain-specific BERT fine-tuning, BERT DAPT	DAT, domain-adaptive Transformer, domain adaptation with Transformers, transfer-learning Transformer
関連	6	2
概要	Domain-adaptive BERT-based classification extends the standard fine-tuning pipeline by first continuing BERT's masked-language-model pre-training on a large corpus of in-domain unlabeled text, then fine-tuning the adapted model on labeled examples for the target classification task. This two-stage approach closes the vocabulary and distributional gap between BERT's general pre-training corpus and specialized domains such as biomedicine, law, finance, or social-media text.	A Domain-Adaptive Transformer (DAT) is a Transformer-based model — such as BERT or ViT — extended with an explicit domain-alignment objective so that learned representations transfer well from a labeled source domain to a different, often unlabeled, target domain. The approach combines the powerful representation capacity of Transformers with domain adaptation techniques such as adversarial training or contrastive alignment to minimise domain shift.