Online Bagging
Also known as: incremental bagging, streaming bagging, online bootstrap aggregating, OzaBag
Online Bagging is a streaming ensemble method introduced by Oza and Russell in 2001 that adapts the classical bootstrap aggregating (Bagging) framework to the online learning setting. Instead of resampling a fixed dataset, each incoming instance is fed to every base learner a Poisson(1)-distributed number of times, faithfully approximating bootstrap sampling as the stream evolves. The result is a robust, incrementally updated ensemble that can handle concept drift and continuous data arrival without storing the entire dataset.
Key highlights
- Processes data one instance at a time with constant memory, enabling truly unbounded streams.
- Poisson weighting provably approximates bootstrap resampling, giving ensemble variance reduction comparable to batch Bagging.
- Naturally parallelisable: each base learner updates independently, making it suitable for distributed streaming architectures.
- No hyperparameter as sensitive as tree depth — the ensemble self-regularises through diversity.
- Extends straightforwardly to multi-class classification and regression without structural changes.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use Online Bagging when data arrives as a continuous stream and storing the full dataset is infeasible or prohibited (e.g., sensor logs, clickstreams, financial tick data, real-time monitoring). It is appropriate when you need classification or regression that updates with every new observation, tolerates mild concept drift, and remains memory-efficient. Do not use Online Bagging as a drop-in replacement for batch Random Forest on a fixed, small-to-medium dataset — batch ensembles with cross-validation will outperform it there. Also avoid it when the data stream exhibits severe abrupt drift without adding an explicit drift-detection mechanism.
Strengths & limitations
- Processes data one instance at a time with constant memory, enabling truly unbounded streams.
- Poisson weighting provably approximates bootstrap resampling, giving ensemble variance reduction comparable to batch Bagging.
- Naturally parallelisable: each base learner updates independently, making it suitable for distributed streaming architectures.
- No hyperparameter as sensitive as tree depth — the ensemble self-regularises through diversity.
- Extends straightforwardly to multi-class classification and regression without structural changes.
- On static datasets, batch methods such as Random Forest or XGBoost consistently outperform Online Bagging because they can use full cross-validation and optimal splits.
- Basic Online Bagging offers only limited resilience to severe concept drift; dedicated drift detectors (e.g., ADWIN) must be added externally.
- Poisson weighting introduces extra variance per instance, which can slow convergence on short streams.
- Interpretability is low: the ensemble of many incrementally grown trees is harder to inspect than a single decision tree.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
Why Poisson(1) specifically?
A Poisson distribution with mean 1 approximates the number of times an observation appears in a bootstrap sample of size n as n approaches infinity. Using Poisson(1) per instance per learner replicates this limiting behaviour exactly, making the online approximation theoretically sound.
How does Online Bagging differ from Adaptive Random Forest?
Adaptive Random Forest (Gomes et al., 2017) extends Online Bagging by adding random feature subsampling at each split (like batch Random Forest) and integrating per-tree drift detectors that reset individual learners when concept drift is detected. Online Bagging is the simpler foundation; Adaptive Random Forest is the current state-of-the-art streaming ensemble built on top of it.
What base learner should I use?
The most common choice is the Hoeffding Tree (also called VFDT), which is the canonical incremental decision tree. It uses Hoeffding bounds to decide when enough data has been seen at a node to split, keeping the tree updatable without storing past instances.
How many base learners do I need?
Empirical benchmarks in the MOA framework suggest 50–100 base learners for stable ensemble performance on most streaming tasks. Fewer learners reduce compute cost but increase prediction variance, especially early in the stream.
How should I evaluate an Online Bagging classifier?
Use prequential evaluation: for each incoming instance, first predict its label, then update the model. Accuracy is computed as the running average of correct predictions. This avoids any look-ahead bias and reflects true deployment conditions.
Sources
- 1.Oza, N. C., & Russell, S. (2001). Online bagging and boosting. In Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics (AISTATS 2001), pp. 105–112.
- 2.Bifet, A., Holmes, G., Kirkby, R., & Pfahringer, B. (2010). MOA: Massive Online Analysis. Journal of Machine Learning Research, 11, 1601–1604.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Online Bagging. ScholarGate. https://scholargate.app/machine-learning/online-bagging