Linganisha mbinu

	Transformer Inayojisimamia Kujifunza	Sentence Embeddings (Vibandiko vya Sentensi)
Nyanja	Ujifunzaji wa Kina	Ujifunzaji wa Kina
Familia	Machine learning	Machine learning
Mwaka wa asili	2017–2019	2015–2019
Mwanzilishi	Vaswani et al. (architecture); Devlin et al. (BERT self-supervised paradigm)	Kiros et al. (Skip-Thought, 2015); Reimers & Gurevych (Sentence-BERT, 2019)
Aina	Self-supervised deep learning model	Representation learning / embedding
Chanzo asilia	Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, 4171–4186.	Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), 3980–3990.
Majina mbadala	SSL Transformer, self-supervised pretraining, masked self-attention pretraining, contrastive transformer	sentence vectors, sentence representations, SBERT, semantic sentence encoding
Zinazohusiana	5	4
Muhtasari	A self-supervised Transformer is a Transformer network pretrained using automatically constructed supervision signals — such as masked token prediction or next-sentence prediction — rather than human-annotated labels. The resulting representations are then fine-tuned or probed on downstream tasks. BERT, GPT, and ViT (Vision Transformer in masked-image modeling mode) are the most widely known instantiations of this paradigm.	Sentence Embeddings convert a sentence or short text into a single fixed-length dense vector that captures its semantic meaning. These vectors allow downstream tasks — semantic similarity, clustering, retrieval, and classification — to operate on numerical representations instead of raw text, making them one of the most versatile building blocks in modern NLP pipelines.