Stacking
Stacked Generalization (Stacking Ensemble with a Meta-Learner) · Also known as: Stacking (Yığınlama — Meta-Öğrenme), stacked generalization, meta-learning ensemble, super learner, yığınlama
Stacking, or stacked generalization, is an ensemble method introduced by David Wolpert in 1992 that combines the outputs of several different base models (Level-0) through a separate meta-model (Level-1). Unlike bagging and boosting, it deliberately uses heterogeneous model types, and it is the standard final-stage strategy in Kaggle competitions.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+7 more
When to use it
Use stacking when predictive accuracy is the priority and you have several reasonably strong, diverse base models to combine, on cross-sectional data with continuous, categorical or binary features and at least about 100 observations. It does not require normally distributed data. Below roughly 100 observations the k-fold cross-validation cannot generate enough reliable Level-0 predictions and the meta-model overfits, so a single tuned model such as random forest or XGBoost is safer.
Strengths & limitations
- Often achieves the highest predictive accuracy by exploiting heterogeneous base models that make different errors.
- Flexible: any mix of model families can serve as base learners, with a simple meta-model on top.
- Assumption-light — no normality requirement, and it handles continuous, categorical and binary features.
- The proven final-stage strategy in competitive prediction settings such as Kaggle.
- Needs a fairly large sample (about 100+) so that k-fold cross-validation can generate reliable Level-0 predictions.
- Gains shrink when the base models are highly correlated rather than diverse.
- More complex and compute-heavy than a single model, and harder to interpret as a whole.
- On small samples the diversity of base models cannot be achieved and a single model is usually enough.
Frequently asked
How is stacking different from bagging and boosting?
Bagging and boosting build many models of the same type, whereas stacking deliberately combines heterogeneous model families and learns how to blend them with a separate meta-model. That diversity is where stacking's advantage comes from.
Why use cross-validation to build the meta-features?
If the meta-model is trained on predictions the base models made on their own training data, information leaks and the stack overfits. Producing Level-0 predictions out-of-fold via k-fold cross-validation gives the meta-model honest inputs.
What should the meta-model be?
Keep it simple — logistic regression, ridge regression or a small XGBoost. A simple meta-learner learns sensible blending weights without overfitting the base-model outputs.
When should I not use stacking?
When you have fewer than about 100 observations, cross-validation cannot produce enough reliable Level-0 predictions and base-model diversity is hard to achieve, so a single tuned random forest or XGBoost is the safer choice.
Sources
- Wolpert, D.H. (1992). Stacked Generalization. Neural Networks, 5(2), 241–259. DOI: 10.1016/S0893-6080(05)80023-1 ↗
- van der Laan, M.J., Polley, E.C. & Hubbard, A.E. (2007). Super Learner. Statistical Applications in Genetics and Molecular Biology, 6(1), Article 25. DOI: 10.2202/1544-6115.1309 ↗
How to cite this page
ScholarGate. (2026, June 1). Stacked Generalization (Stacking Ensemble with a Meta-Learner). ScholarGate. https://scholargate.app/en/machine-learning/stacking-ensemble
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Decision TreeMachine learning↔ compare
- Logistic RegressionResearch Statistics↔ compare
- Random ForestMachine learning↔ compare
- Support Vector MachineMachine learning↔ compare
- XGBoostMachine learning↔ compare