Порівняння методів

	Мультимодальна дифузійна модель	Мультимодальна класифікація на основі BERT
Галузь	Глибоке навчання	Глибоке навчання
Родина	Machine learning	Machine learning
Рік появи	2020–2022	2019
Автор методу	Ho, J. et al. (DDPM); Rombach, R. et al. (LDM/Stable Diffusion)	Kiela, D. et al.; Lu, J. et al.
Тип	Generative model (denoising diffusion)	Multimodal transformer classifier
Основоположне джерело	Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-Resolution Image Synthesis with Latent Diffusion Models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10684–10695.	Kiela, D., Bhooshan, S., Firooz, H., Perez, E., & Testuggine, D. (2019). Supervised multimodal bitransformers for classifying images and text. arXiv preprint arXiv:1909.02950.
Інші назви	multimodal DDPM, cross-modal diffusion, conditional multimodal diffusion, multi-modal denoising diffusion	MMBT, multimodal transformer classification, BERT multimodal fusion, vision-language BERT classifier
Пов'язані	6	2
Підсумок	A multimodal diffusion model extends denoising diffusion probabilistic models to generate or understand content by conditioning on signals from multiple modalities — such as text, image, audio, or video — simultaneously. It learns to reverse a noise process guided by cross-modal context, enabling high-fidelity synthesis and translation across modalities.	Multimodal BERT-based classification extends the BERT transformer architecture to jointly encode and classify data from multiple modalities — most commonly text paired with images — by fusing their representations before a final classification head. Introduced prominently around 2019 through models such as MMBT and ViLBERT, it has become a standard approach for tasks where neither text nor image alone carries sufficient information for accurate labeling.