AdaBoost
AdaBoost (Adaptive Boosting) · Also known as: AdaBoost (Adaptive Boosting), adaptive boosting, adaptif artırma
AdaBoost (Adaptive Boosting) is the original boosting algorithm, introduced by Yoav Freund and Robert Schapire in 1997, that combines a sequence of simple weak learners by giving more weight to the observations they get wrong. The forerunner of gradient boosting, it is simple, interpretable, and a strong baseline for classification.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+2 more
When to use it
A good fit for classification and prediction on cross-sectional tabular data with continuous, categorical, or binary features and at least about 50 observations, when you want a simple, interpretable boosting baseline. AdaBoost makes no normality assumption, but it is sensitive to noisy data and outliers, so screen for outliers first. Control the number of estimators to manage the small but non-zero overfitting risk. On small samples, simpler logistic regression or the more robust Random Forest are safer.
Strengths & limitations
- Simple and interpretable, with few parameters to set.
- Turns weak depth-1 learners into a strong classifier, a solid off-the-shelf baseline.
- No normality assumption; handles continuous, categorical, and binary features.
- The forerunner of gradient boosting, with a clear adaptive-weighting mechanism.
- Sensitive to noisy data and outliers because of its exponential loss; outlier screening is needed.
- Overfitting risk is low but non-zero and must be controlled through the number of estimators.
- On small samples (n below about 50) noisy observations get over-weighted and each iteration can make performance worse.
Frequently asked
How is AdaBoost different from Random Forest?
Random Forest builds many independent trees in parallel and averages them (bagging); AdaBoost builds weak learners sequentially, each focusing on the cases the previous ones got wrong (boosting). Because of its exponential loss, AdaBoost is more sensitive to noise and outliers, while Random Forest is more robust.
Why is AdaBoost sensitive to outliers?
It uses an exponential loss function that penalises misclassification at a sharply increasing rate. Persistently misclassified points — often outliers or noisy observations — keep gaining weight, so they can dominate later iterations and degrade the model.
How many estimators should I use?
The number of estimators (weak learners) controls AdaBoost's small but real overfitting risk. Increase it while validation performance improves, and stop once it plateaus or starts to worsen.
What is a weak learner here?
Typically a decision stump — a decision tree of depth 1. On its own it is only slightly better than chance, but AdaBoost combines many of them, each weighted by its accuracy, into a single strong classifier.
Sources
- Freund, Y. & Schapire, R.E. (1997). A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences, 55(1), 119–139. DOI: 10.1006/jcss.1997.1504 ↗
How to cite this page
ScholarGate. (2026, June 1). AdaBoost (Adaptive Boosting). ScholarGate. https://scholargate.app/en/machine-learning/adaboost
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Decision TreeMachine learning↔ compare
- Logistic RegressionResearch Statistics↔ compare
- Random ForestMachine learning↔ compare
- StackingMachine learning↔ compare
- XGBoostMachine learning↔ compare