Порівняння методів

	Трансформер зі слабким наглядом	Трансформер із самокерованим навчанням
Галузь	Глибоке навчання	Глибоке навчання
Родина	Machine learning	Machine learning
Рік появи	2017–2019	2017–2019
Автор методу	Multiple contributors (weak supervision paradigm: Zhou 2018; transformer backbone: Vaswani et al. 2017)	Vaswani et al. (architecture); Devlin et al. (BERT self-supervised paradigm)
Тип	Weakly supervised deep learning	Self-supervised deep learning model
Основоположне джерело	Ratner, A., Bach, S. H., Ehrenberg, H., Fries, J., Wu, S., & Re, C. (2017). Snorkel: Rapid training data creation with weak supervision. Proceedings of the VLDB Endowment, 11(3), 269–282.	Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, 4171–4186.
Інші назви	WST, weakly supervised attention model, noisy-label transformer, weak supervision with transformers	SSL Transformer, self-supervised pretraining, masked self-attention pretraining, contrastive transformer
Пов'язані	5	5
Підсумок	Weakly Supervised Transformer combines the representational power of Transformer architectures with weak supervision strategies that exploit noisy, incomplete, or programmatically generated labels — making it possible to train high-quality NLP and vision models when fully annotated datasets are scarce or prohibitively expensive to produce.	A self-supervised Transformer is a Transformer network pretrained using automatically constructed supervision signals — such as masked token prediction or next-sentence prediction — rather than human-annotated labels. The resulting representations are then fine-tuned or probed on downstream tasks. BERT, GPT, and ViT (Vision Transformer in masked-image modeling mode) are the most widely known instantiations of this paradigm.