Сравнение методов

	Получение полуобучаемых векторных представлений предложений	Трансформер с полуавтоматическим обучением
Область	Глубокое обучение	Глубокое обучение
Семейство	Machine learning	Machine learning
Год появления	2019–2021	2018–2019
Автор метода	Gao, T.; Reimers, N. et al. (multiple contributors)	Devlin, J. et al. (BERT); broader SSL-Transformer paradigm community
Тип	Semi-supervised representation learning	Semi-supervised deep learning
Основополагающий источник	Gao, T., Yao, X., & Chen, D. (2021). SimCSE: Simple Contrastive Learning of Sentence Embeddings. In Proceedings of EMNLP 2021 (pp. 6894–6910). Association for Computational Linguistics.	Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, 4171–4186.
Другие названия	Semi-supervised SimCSE, Self-training sentence encoders, Pseudo-labeled sentence representation learning, SSL sentence embeddings	semi-supervised transformer model, SSL transformer, transformer with self-supervised pre-training, semi-supervised attention model
Связанные	5	5
Сводка	Semi-supervised sentence embeddings combine a small set of labeled sentence pairs with large quantities of unlabeled text to train dense vector representations of sentences. By exploiting abundant unlabeled data through contrastive objectives or pseudo-labeling, these models produce high-quality embeddings for semantic similarity, retrieval, and classification even when annotated data is scarce.	Semi-supervised learning with Transformer architectures leverages large quantities of unlabeled data alongside a small labeled set to train powerful sequence models. The dominant pattern — exemplified by BERT — first pre-trains the Transformer on unlabeled data using self-supervised objectives such as masked token prediction, then fine-tunes it on the labeled task. This two-stage approach dramatically reduces the labeled data needed to achieve strong performance.