ScholarGate
助手
Machine learning

多头自注意力机制

多头自注意力机制由 Vaswani 及其同事于 2017 年提出，它使序列中的每个位置能够并行计算与其他所有位置的关系。它是 Transformer 架构的核心，也是 BERT、GPT 和 T5 的基础。

在 MethodMind 中打开即将推出视频即将推出Download slides

阅读完整方法

仅限会员

使用免费账户登录即可阅读本节。

登录

Method map

The neighbourhood of related methods — select a node to explore.

多头自注意力机制
BERT微调GPT模型微调LoRA 和 PEFT随机森林XGBoost注意力机制双向循环神经网络检索增强生成（RAG）序列到序列模型

来源

  1. Vaswani, A. et al. (2017). Attention Is All You Need. NeurIPS. link ↗
  2. Devlin, J. et al. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL. link ↗

如何引用本页

ScholarGate. (2026, June 1). Multi-Head Self-Attention (Transformer Core). ScholarGate. https://scholargate.app/zh/deep-learning/self-attention-transformer

Which method?

Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.

Compare side by side →

被引用于

注意力机制双向循环神经网络检索增强生成（RAG）序列到序列模型

发现本页有问题？报告或提出修改建议 →

ScholarGate — Self-Attention (Multi-Head Self-Attention (Transformer Core)). 于 2026-06-15 检索自 https://scholargate.app/zh/deep-learning/self-attention-transformer · 数据集： https://doi.org/10.5281/zenodo.20539026