Segment Anything Model
A Foundation Model for Image Segmentation · Also known as: SAM, Segment Anything
Segment Anything Model (SAM) is a foundation model introduced by Kirillov et al. in 2023 that can segment any object in an image given various forms of prompts. SAM is trained on a massive dataset of diverse images and learns to segment objects based on minimal user input such as points, boxes, or text descriptions.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
SAM is ideal for interactive segmentation and general-purpose object segmentation tasks where different users may request segmentation of different object categories. It excels in scenarios where training task-specific models is impractical due to diversity of potential objects. For specialized domains like medical imaging or satellite analysis, fine-tuning SAM on domain-specific data often outperforms the base model. Standard segmentation networks are preferred when you have large datasets for a specific problem.
Strengths & limitations
- Segments any object in an image with minimal input, avoiding need to train task-specific models
- Robust to diverse object types, textures, and occlusion patterns learned from massive diverse dataset
- Flexible prompting enables interactive segmentation workflows with user refinement
- Fast inference enables real-time interactive applications and rapid prototyping
- May struggle with small objects or fine details; SAM's encoder resolution limits detection of tiny features
- Ambiguous prompts sometimes produce multiple valid masks, requiring user disambiguation
- Performance on specialized domains (medical imaging, satellite) lags behind domain-specific methods until fine-tuning
Frequently asked
What types of prompts does SAM support?
SAM supports points (click on an object), bounding boxes (draw a rectangle), text descriptions, and mask refinement (refine a previous mask). Points can be positive (inside object) or negative (outside object). Multiple prompts can be combined; for ambiguous cases, SAM returns multiple candidate masks ranked by confidence.
How does SAM handle ambiguous cases?
When a prompt could correspond to multiple valid objects (e.g., clicking on a border between two objects), SAM predicts multiple candidate masks. It ranks them using predicted IoU scores, allowing users to select the correct mask. This flexibility enables SAM to handle edge cases without failing.
Can SAM be fine-tuned for specialized domains?
Yes, SAM can be fine-tuned on domain-specific datasets to improve performance. Medical imaging fine-tuning typically improves results significantly. The base model provides a strong initialization that requires less task-specific data than training from scratch, making it practical for specialized applications.
Sources
- Kirillov, A., Mintun, E., Darrell, T., & Girshick, R. (2023). Segment Anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 4015-4026). DOI: 10.1109/iccv51070.2023.00371 ↗
How to cite this page
ScholarGate. (2026, June 3). A Foundation Model for Image Segmentation. ScholarGate. https://scholargate.app/en/deep-learning/segment-anything-model
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- DETR (Detection Transformer)Deep learning↔ compare
- Masked AutoencodersDeep learning↔ compare
- Swin TransformerDeep learning↔ compare
- Vision TransformerDeep learning↔ compare