Machine learningDeep learning / NLP / CV

Multimodal BERT-based Classification

Multimodal BERT-based classification extends the BERT transformer architecture to jointly encode and classify data from multiple modalities — most commonly text paired with images — by fusing their representations before a final classification head. Introduced prominently around 2019 through models such as MMBT and ViLBERT, it has become a standard approach for tasks where neither text nor image alone carries sufficient information for accurate labeling.

MethodMind'de açSoonVideoSoon

Tam yöntemi oku

Members only

Sources

Kiela, D., Bhooshan, S., Firooz, H., Perez, E., & Testuggine, D. (2019). Supervised multimodal bitransformers for classifying images and text. arXiv preprint arXiv:1909.02950. link ↗
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019). ViLBERT: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in Neural Information Processing Systems, 32. link ↗

Related methods

CLIP Vision Transformer

Referenced by

Multimodal Convolutional Neural Network Multimodal Diffusion Model Multimodal Doc2Vec Multimodal Graph Neural Network Multimodal GRU Multimodal Image Classification Multimodal LDA topic model Multimodal Named Entity Recognition Multimodal question answering Multimodal Recurrent Neural Network Multimodal RoBERTa-based Classification Multimodal Text Summarization Multimodal Topic Modeling Multimodal Transformer Multimodal Vision Transformer Multimodal Word2Vec

Bu sayfada bir hata mı var? Bildir / düzeltme öner →