Skip to contentScholarGate
LibraryBookshelfDeskReview StudioAssistant
Sign in
Longformer / BigBird/Evidence
Method evidence record

Longformer / BigBird

Long-sequence Transformers such as Longformer (Beltagy, Peters & Cohan, 2020) and BigBird (Zaheer et al., 2020) replace the standard Transformer's O(n²) attention with sparse attention patterns that scale linearly, O(n), with sequence length. This lets a single model attend over thousands of tokens — full documents, legal texts, or genomic sequences — that would not fit a conventional Transformer.

Sources recorded, not reviewed

Source record

Citations copied verbatim from the method’s source record. No claim-level verification is inferred from them.

Long-Sequence Transformers with Sparse Attention (Longformer / BigBird)
Taxonomic method record · ml-model / deep-learning
  • Beltagy, I., Peters, M. E. & Cohan, A. (2020). Longformer: The Long-Document Transformer. arXiv. · URL
  • Zaheer, M. et al. (2020). Big Bird: Transformers for Longer Sequences. NeurIPS. · URL
Open full method

Curated claims

Claims persisted in the evidence ledger, each with its own assessment.

No curated claims yet

This view does not invent a claim assessment when the ledger has none.

Related methods

Generated from the method graph and shown as machine-suggested relations — no evidence claim is inferred.

Same method familyGraph Attention Networkmachine-suggested · Relational suggestion, not evidence.Same method familyMixture of Expertsmachine-suggested · Relational suggestion, not evidence.Same method familyRandom Forestmachine-suggested · Relational suggestion, not evidence.Same method familyXGBoostmachine-suggested · Relational suggestion, not evidence.

Evidence status

Sources recorded, not reviewed

Bibliographic sources are present. Claim-level evidence review has not been performed.

Sources

2 recorded citations, copied from the method source record.

Actions

Open method page
ScholarGate

A content-first reference library for research methods — what each one is, how it works, and where it comes from.

Open data (CC-BY)

Explore

  • Library
  • Search the library…
  • Browse by field
  • Fields
  • Journey
  • Compare
  • Which method?

Reference

  • Subjects
  • Atlas
  • Glossary
  • Methodology
  • Philosophy

Your tools

  • Bookshelf
  • Desk
  • Chat

Company

  • About
  • Pricing
  • Contact
  • Suggest a method

Entries are compiled from published sources for reference. Verifying the accuracy and suitability of any information for your own use remains your responsibility.

© 2026 ScholarGate · A research-method reference library
  • Privacy
  • Cookies
  • Terms
  • Delete account