Instance Segmentation
Also known as: instance-level segmentation, object instance segmentation, mask prediction, panoptic instance segmentation
Instance segmentation is a computer vision task that simultaneously detects every distinct object in an image and produces a precise pixel-level mask for each individual object instance. Unlike semantic segmentation, which labels every pixel with a class, instance segmentation distinguishes between separate objects of the same class, enabling fine-grained spatial understanding.
Key highlights
- Simultaneously produces bounding boxes, class labels, and pixel-precise masks in a single forward pass.
- Distinguishes individual instances of the same class, enabling accurate object counting and measurement.
- Pre-trained models (e.g. COCO-pretrained Mask R-CNN) transfer well to new domains with moderate fine-tuning.
- Scales to detecting hundreds of instances in a single image.
- Rich ecosystem of frameworks (Detectron2, MMDetection, Ultralytics) with pre-built training pipelines.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use instance segmentation when you need to identify, count, and precisely delineate individual objects within images — such as cells in microscopy, vehicles in aerial imagery, or people in surveillance footage. It excels when objects of the same class overlap or cluster together and must be distinguished. It is not appropriate when only coarse class-level regions are needed (use semantic segmentation instead), when the dataset lacks per-pixel mask annotations (the annotation cost is very high), or when real-time inference on embedded hardware is required without specialised optimisation.
Strengths & limitations
- Simultaneously produces bounding boxes, class labels, and pixel-precise masks in a single forward pass.
- Distinguishes individual instances of the same class, enabling accurate object counting and measurement.
- Pre-trained models (e.g. COCO-pretrained Mask R-CNN) transfer well to new domains with moderate fine-tuning.
- Scales to detecting hundreds of instances in a single image.
- Rich ecosystem of frameworks (Detectron2, MMDetection, Ultralytics) with pre-built training pipelines.
- Requires per-pixel polygon or RLE mask annotations, which are expensive and time-consuming to produce.
- Inference is computationally heavy; real-time performance on standard hardware demands careful model selection and optimisation.
- Performance degrades on heavily occluded or very small objects where boundary detail is ambiguous.
- Large annotated datasets are needed to train from scratch; insufficient data leads to poor mask quality.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What is the difference between instance segmentation and semantic segmentation?
Semantic segmentation assigns a class label to every pixel but does not distinguish between separate objects of the same class — all cars are coloured identically. Instance segmentation goes further: it treats each individual object as a separate entity, so two overlapping cars receive two distinct masks.
How much annotated data do I need?
When fine-tuning a COCO-pretrained model, useful performance can be achieved with a few hundred to a few thousand annotated images per class. Training from scratch typically requires tens of thousands of annotated instances. Annotation tools like CVAT or Labelme can accelerate the process.
Which metric should I report?
The standard metric is mask Average Precision (AP_mask), computed at multiple IoU thresholds (typically 0.50:0.95) following the COCO evaluation protocol. Always report AP_mask rather than only bounding-box AP, and include AP at IoU 0.50 and 0.75 for completeness.
Can instance segmentation run in real time?
Standard Mask R-CNN runs at 5–8 FPS on a GPU, which is insufficient for real-time video. Lightweight alternatives such as YOLACT, SOLOv2, and YOLOv8-seg achieve 30+ FPS with some accuracy trade-off. Model quantisation and TensorRT optimisation further accelerate inference on embedded hardware.
Is instance segmentation suitable for very small objects?
Small-object performance is notoriously difficult. Using high-resolution inputs, FPN multi-scale features, and evaluation on the COCO small-object subset (AP_S) helps diagnose the problem. Specialised architectures or tiling strategies may be needed for domains like satellite imagery with densely packed small objects.
Sources
- 1.He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2961–2969.
- 2.Instance segmentation. Wikipedia.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Instance Segmentation. ScholarGate. https://scholargate.app/deep-learning/instance-segmentation