ScholarGate
助手
Machine learningDeep learning / NLP / CV

多模态Transformer

多模态Transformer扩展了标准的Transformer架构，以处理两种或更多输入模态并对其进行联合推理——最常见的是文本和图像，但也包括音频、视频或结构化数据。跨模态注意力层允许来自一种模态的信息为另一种模态的表示提供依据，从而实现视觉问答、图像字幕生成和多模态情感分析等任务。

在 MethodMind 中打开即将推出视频即将推出Download slides

阅读完整方法

仅限会员

使用免费账户登录即可阅读本节。

登录

Method map

The neighbourhood of related methods — select a node to explore.

多模态Transformer
[需翻译标题：BERT-based Classi…图像分类多模态BERT分类句子嵌入Vision Transformer可解释 Transformer多模态卷积神经网络多模态扩散模型多模态Doc2Vec多模态生成对抗网络

+15 more

来源

  1. Lu, J., Batra, D., Parikh, D., & Lee, S. (2019). ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks. Advances in Neural Information Processing Systems (NeurIPS), 32. link ↗
  2. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. (2021). Learning Transferable Visual Models From Natural Language Supervision. Proceedings of the 38th International Conference on Machine Learning (ICML), PMLR 139. link ↗

如何引用本页

ScholarGate. (2026, June 3). Multimodal Transformer (Cross-Modal Attention-Based Architecture). ScholarGate. https://scholargate.app/zh/deep-learning/multimodal-transformer

Which method?

Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.

Compare side by side →

被引用于

可解释 Transformer多模态卷积神经网络多模态扩散模型多模态Doc2Vec多模态生成对抗网络多模态图神经网络多模态门控循环单元 (Multimodal GRU)多模态图像分类多模态LDA主题模型多模态LSTM多模态多层感知器多模态命名实体识别多模态目标检测多模态问题解答多模态循环神经网络多模态强化学习多模态 RoBERTa 分类多模态文本摘要多模态主题建模多模态Word2Vec

发现本页有问题？报告或提出修改建议 →

ScholarGate — Multimodal Transformer (Multimodal Transformer (Cross-Modal Attention-Based Architecture)). 于 2026-06-15 检索自 https://scholargate.app/zh/deep-learning/multimodal-transformer · 数据集： https://doi.org/10.5281/zenodo.20539026