Swin Transformer
Shifted Window Transformer for Vision · Also known as: Swin, Hierarchical Vision Transformer
The Swin Transformer is a hierarchical vision transformer introduced by Liu et al. in 2021 that uses shifted window attention to achieve computational efficiency while maintaining strong performance on computer vision tasks. Unlike the original Vision Transformer which applies global self-attention, Swin uses local window-based attention with periodic shifting to balance expressiveness and efficiency.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Swin Transformer excels in dense prediction tasks including object detection, semantic segmentation, and instance segmentation, particularly on high-resolution images where global attention becomes computationally prohibitive. It is preferred when computational efficiency is critical while maintaining state-of-the-art accuracy. The hierarchical design makes it suitable as a backbone for downstream tasks. Use alternatives like Vision Transformers for image classification on smaller models or when maximum expressiveness is needed regardless of computation.
Strengths & limitations
- Computational efficiency due to shifted window attention reduces complexity from quadratic to linear in image resolution
- Strong hierarchical representation naturally suits downstream tasks like detection and segmentation
- Achieves state-of-the-art results on ImageNet, COCO, and ADE20K benchmarks
- Flexibility in window size allows tuning the locality-globality tradeoff for specific applications
- Window-based attention may limit long-range dependency learning compared to global attention variants
- Implementation complexity and code optimization requirements exceed simpler CNN baselines
- Performance gains over efficient CNNs may not justify increased implementation burden for some applications
Frequently asked
How does Swin Transformer differ from Vision Transformer?
Vision Transformer applies global self-attention to all patches, resulting in quadratic complexity. Swin uses local window-based attention with periodic shifting, achieving linear complexity while maintaining hierarchical structure like CNNs. Vision Transformer is typically used for classification while Swin excels at dense prediction tasks.
What is the shifted window and why is it necessary?
Windows partition the image into local regions for efficient attention. Shifting windows periodically displaces these boundaries so that adjacent regions can interact. Without shifting, regions separated by a window boundary would never attend to each other, limiting the model's ability to capture global context.
How is the window size chosen?
Window size is a hyperparameter that trades off receptive field size against computation cost. Typical values range from 7 to 14 pixels. Smaller windows reduce computation but limit long-range interactions; larger windows increase computation. The value should be tuned based on the image resolution and available compute.
Sources
- Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021). Swin Transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 10012-10022). DOI: 10.1109/ICCV48922.2021.00986 ↗
How to cite this page
ScholarGate. (2026, June 3). Shifted Window Transformer for Vision. ScholarGate. https://scholargate.app/en/deep-learning/swin-transformer
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- DETR (Detection Transformer)Deep learning↔ compare
- Masked AutoencodersDeep learning↔ compare
- Vision MambaDeep learning↔ compare
- Vision TransformerDeep learning↔ compare