So sánh phương pháp

	GRU tự giám sát	Transformer tự giám sát
Lĩnh vực	Học sâu	Học sâu
Họ	Machine learning	Machine learning
Năm ra đời	2014–2019	2017–2019
Người khởi xướng	Cho, K. et al. (GRU); self-supervised training paradigm from broader SSL literature	Vaswani et al. (architecture); Devlin et al. (BERT self-supervised paradigm)
Loại	Self-supervised sequence model	Self-supervised deep learning model
Công trình gốc	Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In Proceedings of EMNLP 2014.	Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, 4171–4186.
Tên gọi khác	SS-GRU, Self-supervised Gated Recurrent Unit, GRU with self-supervised pretraining, Unsupervised GRU pretraining	SSL Transformer, self-supervised pretraining, masked self-attention pretraining, contrastive transformer
Liên quan	4	5
Tóm tắt	Self-supervised GRU trains a Gated Recurrent Unit network using automatically constructed supervision signals — such as next-step prediction or masked token recovery — derived from the unlabeled data itself. The learned sequence representations are then fine-tuned on small labeled datasets, making high-quality sequential modeling feasible when annotations are scarce.	A self-supervised Transformer is a Transformer network pretrained using automatically constructed supervision signals — such as masked token prediction or next-sentence prediction — rather than human-annotated labels. The resulting representations are then fine-tuned or probed on downstream tasks. BERT, GPT, and ViT (Vision Transformer in masked-image modeling mode) are the most widely known instantiations of this paradigm.