Weakly Supervised Instance Segmentation
Also known as: WSIS, weakly-supervised mask prediction, weak-label instance segmentation, box-supervised instance segmentation
Weakly supervised instance segmentation trains deep networks to delineate individual object instances at pixel level using only cheap, incomplete annotations — such as bounding boxes, image-level labels, or point clicks — rather than costly full pixel-wise masks. It dramatically reduces annotation effort while still producing instance-level masks for each object in an image.
Key highlights
- Drastically reduces annotation cost by replacing pixel masks with bounding boxes or image-level labels.
- Compatible with standard instance segmentation architectures (Mask R-CNN, YOLACT) by replacing the mask supervision signal.
- Self-training with pseudo-labels allows iterative quality improvement without additional human annotation.
- Applicable to domains where full mask annotation is practically infeasible at scale.
- Generalises well when combined with strong visual priors such as CRF-based boundary regularisation.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use weakly supervised instance segmentation when you have a large collection of images requiring instance-level masks but annotation budget is limited — scenarios where bounding-box annotations (which are 5–10 times faster to collect than polygon masks) are feasible. Ideal for medical imaging, satellite imagery, industrial inspection, or any domain where mask annotation is a bottleneck. Do not use when high boundary precision is critical and mask-annotated data is available in sufficient quantity; fully supervised methods such as Mask R-CNN will then outperform any weakly supervised approach. Also avoid if the objects are highly irregular or heavily occluded, as weak signals may not provide enough spatial guidance.
Strengths & limitations
- Drastically reduces annotation cost by replacing pixel masks with bounding boxes or image-level labels.
- Compatible with standard instance segmentation architectures (Mask R-CNN, YOLACT) by replacing the mask supervision signal.
- Self-training with pseudo-labels allows iterative quality improvement without additional human annotation.
- Applicable to domains where full mask annotation is practically infeasible at scale.
- Generalises well when combined with strong visual priors such as CRF-based boundary regularisation.
- Mask quality consistently lags behind fully supervised methods, especially on thin structures and occluded boundaries.
- Box-tightness priors assume each box is axis-aligned and tight, an assumption that fails for rotated or loosely annotated boxes.
- Image-level supervision provides no spatial guidance for overlapping instances of the same class.
- Self-training can amplify early errors, causing error propagation across pseudo-label rounds.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What is the most common form of weak supervision for instance segmentation?
Bounding-box annotations are the dominant weak signal because they are easy to collect, spatially informative, and compatible with tightness-prior losses. Image-level labels are cheaper but provide far less spatial guidance and struggle with multi-instance scenes.
How much accuracy do I lose compared to fully supervised training?
On standard benchmarks such as COCO, box-supervised methods typically trail fully supervised Mask R-CNN by 5–15 mask-IoU points, depending on the dataset and refinement strategy. The gap has been narrowing with transformer-based approaches and pseudo-label self-training.
Can I combine weak and strong labels?
Yes. Semi-supervised instance segmentation uses a small set of fully annotated images alongside a larger weakly annotated set. This hybrid often substantially outperforms purely weakly supervised approaches when even a few hundred mask-annotated samples are available.
Does weak supervision work for heavily occluded or overlapping instances?
This is the hardest setting for any weak supervision approach. Image-level labels cannot distinguish individual overlapping instances. Box-supervised methods can partially handle occlusion if boxes are annotated per instance, but boundary quality degrades significantly in dense, heavily overlapping scenes.
What evaluation metrics should I report?
Report mask Average Precision at multiple IoU thresholds (AP50, AP75, AP) following the COCO protocol. Additionally report Boundary IoU or the F-measure for boundary accuracy, as standard mask IoU can overestimate contour quality in weakly supervised settings.
Sources
- 1.Hsu, C.-C., Hsu, K.-J., Tsai, C.-C., Lin, Y.-Y., & Chuang, Y.-Y. (2019). Weakly supervised instance segmentation using the bounding box tightness prior. Advances in Neural Information Processing Systems (NeurIPS), 32.
- 2.Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning deep features for discriminative localization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2921–2929.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Weakly Supervised Instance Segmentation. ScholarGate. https://scholargate.app/deep-learning/weakly-supervised-instance-segmentation