השוואת שיטות

	סיווג תמונות מולטימודאלי	שיכוני משפטים מולטי-מודאליים
תחום	למידה עמוקה	למידה עמוקה
משפחה	Machine learning	Machine learning
שנת המקור	2011–2021	2013–2021
הוגה השיטה	Ngiam et al.; Radford et al. (CLIP)	Frome et al. (DeViSE, 2013); popularized by Radford et al. (CLIP, 2021)
סוג	Multimodal supervised classification	Representation learning model
מקור מכונן	Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. Proceedings of the 38th International Conference on Machine Learning (ICML), PMLR 139, 8748–8763.	Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (ICML), pp. 8748–8763. PMLR.
כינויים	multimodal visual classification, image-text classification, vision-language classification, cross-modal image classification	multimodal embeddings, cross-modal sentence embeddings, vision-language embeddings, joint image-text embeddings
קשורות	6	1
תקציר	Multimodal image classification extends standard visual classification by incorporating additional modalities — such as text captions, audio, or structured metadata — alongside image features. Separate encoders process each modality, their representations are fused, and a joint classifier assigns the target label. Models such as CLIP demonstrate that image–text alignment enables zero-shot and few-shot image classification at scale.	Multimodal sentence embeddings map text and images (and sometimes audio or video) into a shared continuous vector space, so that semantically related pairs from different modalities land close together. Trained by contrastive objectives on large paired corpora, these representations power cross-modal retrieval, zero-shot classification, and vision-language reasoning.