Online Gradient Boosting
Online Gradient Boosting (Streaming Gradient Boosted Ensembles) · Also known as: OGB, streaming gradient boosting, incremental gradient boosting, online boosting with gradient descent
Online Gradient Boosting adapts the gradient boosting framework for streaming settings where data arrives one sample at a time rather than as a fixed batch. At each step the model computes a pseudo-residual for the incoming observation and updates a weak learner in place, growing an additive ensemble without storing or revisiting past data. This makes it suitable for real-time prediction and large-scale streaming pipelines where retraining from scratch is infeasible.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Online Gradient Boosting when data arrives as a continuous stream and retraining from scratch after each new batch is too slow or impractical, when storage of historical data is infeasible due to volume or privacy constraints, or when the data-generating process may shift over time (concept drift) and the model must adapt incrementally. It is also appropriate for very large datasets where a single batch pass is prohibitively expensive. Do NOT use it when the full dataset fits comfortably in memory and computational resources allow standard batch gradient boosting — batch methods such as XGBoost or LightGBM will generally achieve higher accuracy under those conditions. Avoid it when the number of boosting rounds cannot be fixed in advance or when interpretability of individual trees is a hard requirement.
Strengths & limitations
- Processes data in a single pass without storing the full dataset, making it memory-efficient for large or infinite streams.
- Adapts to concept drift because recent samples continuously update the weak learners.
- Retains the strong predictive power of gradient boosting in settings where batch training is infeasible.
- Predictions can be served in real time at each step — the model is always ready to predict.
- Compatible with any differentiable loss function, supporting regression, binary and multiclass classification.
- Generally achieves lower accuracy than batch gradient boosting (XGBoost, LightGBM) when all data are available, because each sample is seen only once.
- The number of boosting rounds must be fixed before streaming starts, limiting structural flexibility.
- Sensitive to the order of arriving samples: shuffling or distribution shifts in the stream can destabilise training.
- Convergence guarantees are weaker than in the batch setting and depend heavily on the chosen learning rate.
Frequently asked
How does Online Gradient Boosting differ from standard XGBoost or LightGBM?
XGBoost and LightGBM are batch methods that iterate over the complete training set multiple times to fit trees greedily. Online Gradient Boosting updates its weak learners after each single observation and never revisits past data. Batch methods are more accurate when data fit in memory; the online variant trades accuracy for scalability and real-time adaptability.
How should I evaluate an online model fairly?
Use prequential (test-then-train) evaluation: for each incoming sample, record the model's prediction before updating it with that sample's label. This produces an unbiased, temporally honest error estimate without requiring a held-out test set.
How do I handle concept drift?
Pair Online Gradient Boosting with a drift-detection algorithm (e.g., ADWIN or DDM) that signals when the error rate rises significantly. On detection, reset or reinitialise the affected weak learners so the ensemble can re-adapt to the new distribution.
Is Online Gradient Boosting suitable for multiclass problems?
Yes. With a multiclass differentiable loss (e.g., softmax cross-entropy), the pseudo-residual becomes a vector of class-specific errors and each weak learner is updated to correct all class outputs simultaneously, exactly as in the batch multiclass gradient boosting formulation.
What weak learners work best in practice?
Shallow Hoeffding trees (which themselves support online splits) or online linear models are the most common choices. They update efficiently with each new sample and avoid the need to store data for a batch refit at each boosting step.
Sources
- Grubb, A. & Bagnell, J. A. (2011). Generalized Boosting Algorithms for Convex Optimization. Proceedings of the 28th International Conference on Machine Learning (ICML 2011), 1209–1216. link ↗
- Beygelzimer, A., Hazan, E., Langford, J. & Zheng, T. (2015). Online-to-Batch Conversions and Applications. Advances in Neural Information Processing Systems (NeurIPS), 28. link ↗
How to cite this page
ScholarGate. (2026, June 3). Online Gradient Boosting (Streaming Gradient Boosted Ensembles). ScholarGate. https://scholargate.app/en/machine-learning/online-gradient-boosting
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- BoostingMachine learning↔ compare
- Gradient BoostingMachine learning↔ compare
- Online LearningMachine learning↔ compare
- Online Random ForestMachine learning↔ compare
- Semi-supervised Gradient BoostingMachine learning↔ compare
- XGBoostMachine learning↔ compare