Machine learning
长格式Transformer / BigBird
长序列Transformer，如Longformer（Beltagy, Peters & Cohan, 2020）和BigBird（Zaheer et al., 2020），用稀疏注意力模式取代了标准Transformer的O(n²)注意力，使其能以O(n)的线性复杂度随序列长度扩展。这使得单个模型能够处理数千个token——完整的文档、法律文本或基因序列——而这些内容是传统Transformer无法容纳的。
阅读完整方法
仅限会员
登录使用免费账户登录即可阅读本节。
Method map
The neighbourhood of related methods — select a node to explore.
来源
如何引用本页
ScholarGate. (2026, June 1). Long-Sequence Transformers with Sparse Attention (Longformer / BigBird). ScholarGate. https://scholargate.app/zh/deep-learning/longformer-bigbird
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
Compare side by side →