Background Subtraction
Background Subtraction for Foreground Detection · Also known as: Foreground detection, Video segmentation
Background subtraction is a video processing technique that separates moving foreground objects from a static or slowly changing background by comparing each frame to a learned or estimated background model. Widely used in video surveillance and motion detection, background subtraction enables robust foreground detection even in complex scenes with illumination changes.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use background subtraction for video surveillance, motion detection, and object tracking in scenes where background is relatively stable or changes slowly and predictably. Background subtraction works well for outdoor scenes where background changes gradually due to weather or time of day. Avoid background subtraction when background is highly dynamic (water, waving trees) or when foreground and background are difficult to distinguish (camouflaged objects); consider optical flow or deep learning alternatives.
Strengths & limitations
- Computationally efficient; runs in real-time on standard hardware
- Adapts to gradual background changes (illumination, weather) without manual retraining
- Requires no object model or training data; learns from the video itself
- Detects all moving objects without target-specific recognition
- Works across diverse video domains (surveillance, traffic, sports)
- Fails when background is dynamic (ocean waves, moving trees, flickering lights)
- Sensitive to shadows and reflections; incorrectly classifies shadows as foreground
- Sudden global illumination changes (light switch, sunrise) can corrupt the background model
- Slow-moving or stationary objects may be absorbed into the background model over time
Frequently asked
Why use a Gaussian Mixture Model (GMM) instead of a single Gaussian for each pixel?
Background pixels exhibit natural variation: surfaces reflect light at slightly different angles, causing intensity fluctuations; vegetation and water move subtly. A single Gaussian cannot capture this multi-modal behavior. A mixture of Gaussians allows some components to represent stable background (low variance) and others to represent non-foreground variations (low confidence).
How does background subtraction adapt to illumination changes?
The background model is updated continuously: when a pixel matches the background model, that observation is used to adjust the Gaussian components (mean and variance). This incremental learning allows the model to drift gradually toward new illumination conditions. However, sudden changes (like turning on a light) can overwhelm the model until enough frames have been processed.
What is shadow detection and why is it necessary?
Shadows are regions where foreground objects block light from the background, darkening that area. Naive background subtraction classifies shadows as foreground because their intensity is different from the background. Shadow detection identifies regions where intensity decreased proportionally (shadow signature) and classifies them as background, improving foreground object boundaries.
How does background subtraction handle temporary occlusions or re-lighting?
Occlusions (a person standing still) gradually become absorbed into the background model because the area matches the model's Gaussian distribution for enough frames. To prevent this, some methods explicitly track persistent foreground regions or use a separate 'shadow' component for long-term occlusion. Re-lighting is handled by model adaptation, though sudden changes still cause false detections until the model adjusts.
Sources
- Stauffer, C., & Grimson, W. E. L. (1999). Adaptive background mixture models for real-time tracking. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 246–252. DOI: 10.1109/CVPR.1999.784637 ↗
- KaewTraKulPong, P., & Bowden, R. (2002). An improved adaptive background mixture model for real-time tracking with shadow detection. Proceedings of the European Conference on Computer Vision (ECCV), 135–144. link ↗
How to cite this page
ScholarGate. (2026, June 3). Background Subtraction for Foreground Detection. ScholarGate. https://scholargate.app/en/computer-vision/background-subtraction
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Canny Edge DetectionComputer Vision↔ compare
- Contour AnalysisComputer Vision↔ compare
- Histogram EqualizationComputer Vision↔ compare
- Image Morphology OperationsComputer Vision↔ compare
- Watershed SegmentationComputer Vision↔ compare