Metodebevisregistrering
Mixture of Experts
Mixture of Experts (MoE) is a sparse neural-network architecture, introduced by Shazeer and colleagues in 2017 with the sparsely-gated MoE layer, in which only a subset of expert sub-networks is activated for each input. As seen in models such as Switch Transformer and Mixtral, it holds computation cost fixed even as the total parameter count grows.
Kilderegistrering
Citater kopieret ordret fra metodens kilderegistrering. Ingen påstandsniveauverifikation er udledt heraf.
Sparsely-Gated Mixture of Experts (MoE)
Taksonomisk metoderegistrering · ml-model / deep-learning
- Shazeer, N. et al. (2017). Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. ICLR. arXiv:1701.06538 · URL
- Jiang, A.Q. et al. (2024). Mixtral of Experts. arXiv. · URL
Kuraterede påstande
Påstande gemt i bevis-loggen, hver med sin egen vurdering.
Ingen kuraterede påstande endnu
Denne visning opfinder ikke en påstandsvurdering, når loggen ingen har.
Relaterede metoder
Genereret fra metodegrafen og vist som maskinelt foreslåede relationer — ingen bevispåstand er udledt.