So sánh phương pháp

	Transformer đa ngôn ngữ	Phân loại dựa trên RoBERTa
Lĩnh vực	Học sâu	Học sâu
Họ	Machine learning	Machine learning
Năm ra đời	2019–2020	2019
Người khởi xướng	Devlin et al. (mBERT); Conneau et al. (XLM-R)	Liu, Y. et al. (Facebook AI Research / University of Washington)
Loại	Pre-trained cross-lingual language model	Pre-trained transformer fine-tuned for sequence classification
Công trình gốc	Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, pp. 4171–4186. Association for Computational Linguistics.	Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692.
Tên gọi khác	multilingual LM, cross-lingual transformer, mBERT-style model, multilingual pre-trained model	RoBERTa classifier, RoBERTa text classification, Robustly Optimized BERT Classification, RoBERTa fine-tuning for classification
Liên quan	4	5
Tóm tắt	A multilingual transformer is a pre-trained language model built on the transformer architecture and trained jointly on text from dozens to over one hundred languages. Models such as mBERT and XLM-RoBERTa learn shared cross-lingual representations, enabling zero-shot or few-shot transfer: a model fine-tuned on English data can often be applied directly to French, German, Arabic, or Chinese without language-specific labels.	RoBERTa-based Classification applies the RoBERTa pre-trained transformer — trained more robustly than BERT with dynamic masking and larger batches — to text categorisation tasks by adding a lightweight classification head on top of the [CLS] token representation and fine-tuning the entire model on labelled examples. It consistently matches or outperforms BERT on standard NLP benchmarks.