Online Bagging
Online Bagging (Incremental Bootstrap Aggregating) · Also known as: incremental bagging, streaming bagging, online bootstrap aggregating, OzaBag
Online Bagging is a streaming ensemble method introduced by Oza and Russell in 2001 that adapts the classical bootstrap aggregating (Bagging) framework to the online learning setting. Instead of resampling a fixed dataset, each incoming instance is fed to every base learner a Poisson(1)-distributed number of times, faithfully approximating bootstrap sampling as the stream evolves. The result is a robust, incrementally updated ensemble that can handle concept drift and continuous data arrival without storing the entire dataset.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Online Bagging when data arrives as a continuous stream and storing the full dataset is infeasible or prohibited (e.g., sensor logs, clickstreams, financial tick data, real-time monitoring). It is appropriate when you need classification or regression that updates with every new observation, tolerates mild concept drift, and remains memory-efficient. Do not use Online Bagging as a drop-in replacement for batch Random Forest on a fixed, small-to-medium dataset — batch ensembles with cross-validation will outperform it there. Also avoid it when the data stream exhibits severe abrupt drift without adding an explicit drift-detection mechanism.
Strengths & limitations
- Processes data one instance at a time with constant memory, enabling truly unbounded streams.
- Poisson weighting provably approximates bootstrap resampling, giving ensemble variance reduction comparable to batch Bagging.
- Naturally parallelisable: each base learner updates independently, making it suitable for distributed streaming architectures.
- No hyperparameter as sensitive as tree depth — the ensemble self-regularises through diversity.
- Extends straightforwardly to multi-class classification and regression without structural changes.
- On static datasets, batch methods such as Random Forest or XGBoost consistently outperform Online Bagging because they can use full cross-validation and optimal splits.
- Basic Online Bagging offers only limited resilience to severe concept drift; dedicated drift detectors (e.g., ADWIN) must be added externally.
- Poisson weighting introduces extra variance per instance, which can slow convergence on short streams.
- Interpretability is low: the ensemble of many incrementally grown trees is harder to inspect than a single decision tree.
Frequently asked
Why Poisson(1) specifically?
A Poisson distribution with mean 1 approximates the number of times an observation appears in a bootstrap sample of size n as n approaches infinity. Using Poisson(1) per instance per learner replicates this limiting behaviour exactly, making the online approximation theoretically sound.
How does Online Bagging differ from Adaptive Random Forest?
Adaptive Random Forest (Gomes et al., 2017) extends Online Bagging by adding random feature subsampling at each split (like batch Random Forest) and integrating per-tree drift detectors that reset individual learners when concept drift is detected. Online Bagging is the simpler foundation; Adaptive Random Forest is the current state-of-the-art streaming ensemble built on top of it.
What base learner should I use?
The most common choice is the Hoeffding Tree (also called VFDT), which is the canonical incremental decision tree. It uses Hoeffding bounds to decide when enough data has been seen at a node to split, keeping the tree updatable without storing past instances.
How many base learners do I need?
Empirical benchmarks in the MOA framework suggest 50–100 base learners for stable ensemble performance on most streaming tasks. Fewer learners reduce compute cost but increase prediction variance, especially early in the stream.
How should I evaluate an Online Bagging classifier?
Use prequential evaluation: for each incoming instance, first predict its label, then update the model. Accuracy is computed as the running average of correct predictions. This avoids any look-ahead bias and reflects true deployment conditions.
Sources
- Oza, N. C., & Russell, S. (2001). Online bagging and boosting. In Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics (AISTATS 2001), pp. 105–112. link ↗
- Bifet, A., Holmes, G., Kirkby, R., & Pfahringer, B. (2010). MOA: Massive Online Analysis. Journal of Machine Learning Research, 11, 1601–1604. link ↗
How to cite this page
ScholarGate. (2026, June 3). Online Bagging (Incremental Bootstrap Aggregating). ScholarGate. https://scholargate.app/en/machine-learning/online-bagging
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- BaggingMachine learning↔ compare
- Gradient BoostingMachine learning↔ compare
- Online BoostingMachine learning↔ compare
- Random ForestMachine learning↔ compare