Methoden vergleichen

	Schwach überwachter Transformer	Selbstüberwachter Transformer
Fachgebiet	Deep Learning	Deep Learning
Familie	Machine learning	Machine learning
Entstehungsjahr	2017–2019	2017–2019
Urheber	Multiple contributors (weak supervision paradigm: Zhou 2018; transformer backbone: Vaswani et al. 2017)	Vaswani et al. (architecture); Devlin et al. (BERT self-supervised paradigm)
Typ	Weakly supervised deep learning	Self-supervised deep learning model
Wegweisende Quelle	Ratner, A., Bach, S. H., Ehrenberg, H., Fries, J., Wu, S., & Re, C. (2017). Snorkel: Rapid training data creation with weak supervision. Proceedings of the VLDB Endowment, 11(3), 269–282.	Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, 4171–4186.
Aliasnamen	WST, weakly supervised attention model, noisy-label transformer, weak supervision with transformers	SSL Transformer, self-supervised pretraining, masked self-attention pretraining, contrastive transformer
Verwandt	5	5
Zusammenfassung	Weakly Supervised Transformer combines the representational power of Transformer architectures with weak supervision strategies that exploit noisy, incomplete, or programmatically generated labels — making it possible to train high-quality NLP and vision models when fully annotated datasets are scarce or prohibitively expensive to produce.	A self-supervised Transformer is a Transformer network pretrained using automatically constructed supervision signals — such as masked token prediction or next-sentence prediction — rather than human-annotated labels. The resulting representations are then fine-tuned or probed on downstream tasks. BERT, GPT, and ViT (Vision Transformer in masked-image modeling mode) are the most widely known instantiations of this paradigm.