Instance Segmentation
Instance Segmentation (per-object pixel-level detection and masking) · Also known as: instance-level segmentation, object instance segmentation, mask prediction, panoptic instance segmentation
Instance segmentation is a computer vision task that simultaneously detects every distinct object in an image and produces a precise pixel-level mask for each individual object instance. Unlike semantic segmentation, which labels every pixel with a class, instance segmentation distinguishes between separate objects of the same class, enabling fine-grained spatial understanding.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+9 more
When to use it
Use instance segmentation when you need to identify, count, and precisely delineate individual objects within images — such as cells in microscopy, vehicles in aerial imagery, or people in surveillance footage. It excels when objects of the same class overlap or cluster together and must be distinguished. It is not appropriate when only coarse class-level regions are needed (use semantic segmentation instead), when the dataset lacks per-pixel mask annotations (the annotation cost is very high), or when real-time inference on embedded hardware is required without specialised optimisation.
Strengths & limitations
- Simultaneously produces bounding boxes, class labels, and pixel-precise masks in a single forward pass.
- Distinguishes individual instances of the same class, enabling accurate object counting and measurement.
- Pre-trained models (e.g. COCO-pretrained Mask R-CNN) transfer well to new domains with moderate fine-tuning.
- Scales to detecting hundreds of instances in a single image.
- Rich ecosystem of frameworks (Detectron2, MMDetection, Ultralytics) with pre-built training pipelines.
- Requires per-pixel polygon or RLE mask annotations, which are expensive and time-consuming to produce.
- Inference is computationally heavy; real-time performance on standard hardware demands careful model selection and optimisation.
- Performance degrades on heavily occluded or very small objects where boundary detail is ambiguous.
- Large annotated datasets are needed to train from scratch; insufficient data leads to poor mask quality.
Frequently asked
What is the difference between instance segmentation and semantic segmentation?
Semantic segmentation assigns a class label to every pixel but does not distinguish between separate objects of the same class — all cars are coloured identically. Instance segmentation goes further: it treats each individual object as a separate entity, so two overlapping cars receive two distinct masks.
How much annotated data do I need?
When fine-tuning a COCO-pretrained model, useful performance can be achieved with a few hundred to a few thousand annotated images per class. Training from scratch typically requires tens of thousands of annotated instances. Annotation tools like CVAT or Labelme can accelerate the process.
Which metric should I report?
The standard metric is mask Average Precision (AP_mask), computed at multiple IoU thresholds (typically 0.50:0.95) following the COCO evaluation protocol. Always report AP_mask rather than only bounding-box AP, and include AP at IoU 0.50 and 0.75 for completeness.
Can instance segmentation run in real time?
Standard Mask R-CNN runs at 5–8 FPS on a GPU, which is insufficient for real-time video. Lightweight alternatives such as YOLACT, SOLOv2, and YOLOv8-seg achieve 30+ FPS with some accuracy trade-off. Model quantisation and TensorRT optimisation further accelerate inference on embedded hardware.
Is instance segmentation suitable for very small objects?
Small-object performance is notoriously difficult. Using high-resolution inputs, FPN multi-scale features, and evaluation on the COCO small-object subset (AP_S) helps diagnose the problem. Specialised architectures or tiling strategies may be needed for domains like satellite imagery with densely packed small objects.
Sources
- He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2961–2969. DOI: 10.1109/ICCV.2017.322 ↗
- Instance segmentation. Wikipedia. link ↗
How to cite this page
ScholarGate. (2026, June 3). Instance Segmentation (per-object pixel-level detection and masking). ScholarGate. https://scholargate.app/en/deep-learning/instance-segmentation
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Image ClassificationDeep learning↔ compare
- Multimodal Instance SegmentationDeep learning↔ compare
- Object DetectionDeep learning↔ compare
- Semantic SegmentationDeep learning↔ compare