ScholarGate
Assistent
Machine learning

Multi-Head Self-Attention

Multi-head self-attention, introduceret af Vaswani og kolleger i 2017, er den mekanisme, der lader hver position i en sekvens beregne sin relation til alle andre positioner parallelt. Den udgør kernen i Transformer-arkitekturen og fundamentet for BERT, GPT og T5.

Åbn i MethodMindSnartVideoSnartDownload slides

Læs hele metoden

Kun for medlemmer

Log ind med en gratis konto for at læse dette afsnit.

Log ind

Method map

The neighbourhood of related methods — select a node to explore.

Multi-Head Self-Attention
BERT FinjusteringGPT FinjusteringLoRA og PEFTRandom ForestXGBoostOpmærksomhedsmekanismeBidirectional RNNRetrieval-Augmented Gene…Sekvens-til-sekvens-model

Kilder

  1. Vaswani, A. et al. (2017). Attention Is All You Need. NeurIPS. link ↗
  2. Devlin, J. et al. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL. link ↗

Sådan citerer du denne side

ScholarGate. (2026, June 1). Multi-Head Self-Attention (Transformer Core). ScholarGate. https://scholargate.app/da/deep-learning/self-attention-transformer

Which method?

Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.

Compare side by side →

Refereret af

OpmærksomhedsmekanismeBidirectional RNNRetrieval-Augmented Generation (RAG)Sekvens-til-sekvens-model

Har du fundet en fejl på denne side? Indberet den eller foreslå en rettelse →

ScholarGate — Self-Attention (Multi-Head Self-Attention (Transformer Core)). Hentet 2026-06-15 fra https://scholargate.app/da/deep-learning/self-attention-transformer · Datasæt: https://doi.org/10.5281/zenodo.20539026