手法を比較

	ドメイン適応型BERTベース分類	RoBERTaベースの分類
分野	深層学習	深層学習
系統	Machine learning	Machine learning
提唱年	2019–2020	2019
提唱者	Gururangan et al. (2020); earlier domain-specific instances include Lee et al. (2020) — BioBERT	Liu, Y. et al. (Facebook AI Research / University of Washington)
種類	Domain-adaptive pre-training followed by supervised fine-tuning	Pre-trained transformer fine-tuned for sequence classification
原典	Gururangan, S., Marasovic, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020). Don't Stop Pretraining: Adapt Language Models to Domains and Tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020), 8342–8360.	Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692.
別名	DAPT BERT classification, domain-adaptive pre-training, domain-specific BERT fine-tuning, BERT DAPT	RoBERTa classifier, RoBERTa text classification, Robustly Optimized BERT Classification, RoBERTa fine-tuning for classification
関連	6	5
概要	Domain-adaptive BERT-based classification extends the standard fine-tuning pipeline by first continuing BERT's masked-language-model pre-training on a large corpus of in-domain unlabeled text, then fine-tuning the adapted model on labeled examples for the target classification task. This two-stage approach closes the vocabulary and distributional gap between BERT's general pre-training corpus and specialized domains such as biomedicine, law, finance, or social-media text.	RoBERTa-based Classification applies the RoBERTa pre-trained transformer — trained more robustly than BERT with dynamic masking and larger batches — to text categorisation tasks by adding a lightweight classification head on top of the [CLS] token representation and fine-tuning the entire model on labelled examples. It consistently matches or outperforms BERT on standard NLP benchmarks.