Weakly Supervised Instance Segmentation
Weakly Supervised Instance Segmentation (Deep Learning with Incomplete Annotations) · Also known as: WSIS, weakly-supervised mask prediction, weak-label instance segmentation, box-supervised instance segmentation
Weakly supervised instance segmentation trains deep networks to delineate individual object instances at pixel level using only cheap, incomplete annotations — such as bounding boxes, image-level labels, or point clicks — rather than costly full pixel-wise masks. It dramatically reduces annotation effort while still producing instance-level masks for each object in an image.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use weakly supervised instance segmentation when you have a large collection of images requiring instance-level masks but annotation budget is limited — scenarios where bounding-box annotations (which are 5–10 times faster to collect than polygon masks) are feasible. Ideal for medical imaging, satellite imagery, industrial inspection, or any domain where mask annotation is a bottleneck. Do not use when high boundary precision is critical and mask-annotated data is available in sufficient quantity; fully supervised methods such as Mask R-CNN will then outperform any weakly supervised approach. Also avoid if the objects are highly irregular or heavily occluded, as weak signals may not provide enough spatial guidance.
Strengths & limitations
- Drastically reduces annotation cost by replacing pixel masks with bounding boxes or image-level labels.
- Compatible with standard instance segmentation architectures (Mask R-CNN, YOLACT) by replacing the mask supervision signal.
- Self-training with pseudo-labels allows iterative quality improvement without additional human annotation.
- Applicable to domains where full mask annotation is practically infeasible at scale.
- Generalises well when combined with strong visual priors such as CRF-based boundary regularisation.
- Mask quality consistently lags behind fully supervised methods, especially on thin structures and occluded boundaries.
- Box-tightness priors assume each box is axis-aligned and tight, an assumption that fails for rotated or loosely annotated boxes.
- Image-level supervision provides no spatial guidance for overlapping instances of the same class.
- Self-training can amplify early errors, causing error propagation across pseudo-label rounds.
Frequently asked
What is the most common form of weak supervision for instance segmentation?
Bounding-box annotations are the dominant weak signal because they are easy to collect, spatially informative, and compatible with tightness-prior losses. Image-level labels are cheaper but provide far less spatial guidance and struggle with multi-instance scenes.
How much accuracy do I lose compared to fully supervised training?
On standard benchmarks such as COCO, box-supervised methods typically trail fully supervised Mask R-CNN by 5–15 mask-IoU points, depending on the dataset and refinement strategy. The gap has been narrowing with transformer-based approaches and pseudo-label self-training.
Can I combine weak and strong labels?
Yes. Semi-supervised instance segmentation uses a small set of fully annotated images alongside a larger weakly annotated set. This hybrid often substantially outperforms purely weakly supervised approaches when even a few hundred mask-annotated samples are available.
Does weak supervision work for heavily occluded or overlapping instances?
This is the hardest setting for any weak supervision approach. Image-level labels cannot distinguish individual overlapping instances. Box-supervised methods can partially handle occlusion if boxes are annotated per instance, but boundary quality degrades significantly in dense, heavily overlapping scenes.
What evaluation metrics should I report?
Report mask Average Precision at multiple IoU thresholds (AP50, AP75, AP) following the COCO protocol. Additionally report Boundary IoU or the F-measure for boundary accuracy, as standard mask IoU can overestimate contour quality in weakly supervised settings.
Sources
- Hsu, C.-C., Hsu, K.-J., Tsai, C.-C., Lin, Y.-Y., & Chuang, Y.-Y. (2019). Weakly supervised instance segmentation using the bounding box tightness prior. Advances in Neural Information Processing Systems (NeurIPS), 32. link ↗
- Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning deep features for discriminative localization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2921–2929. DOI: 10.1109/CVPR.2016.319 ↗
How to cite this page
ScholarGate. (2026, June 3). Weakly Supervised Instance Segmentation (Deep Learning with Incomplete Annotations). ScholarGate. https://scholargate.app/en/deep-learning/weakly-supervised-instance-segmentation
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Instance SegmentationDeep learning↔ compare
- Object DetectionDeep learning↔ compare
- Self-supervised Instance SegmentationDeep learning↔ compare
- Semantic SegmentationDeep learning↔ compare
- Semi-supervised Instance SegmentationDeep learning↔ compare
- Weakly Supervised Semantic SegmentationDeep learning↔ compare