Semi-supervised Bagging
Semi-supervised Bagging (Bootstrap Aggregating with Unlabeled Data) · Also known as: SS-Bagging, semi-supervised bootstrap aggregating, self-training bagging, bagging with pseudo-labels
Semi-supervised Bagging extends the classical bagging ensemble to settings where labeled training examples are scarce but large amounts of unlabeled data are available. Base learners trained on labeled data assign pseudo-labels to unlabeled examples; the expanded dataset is then used to grow a diverse ensemble whose aggregated vote is more accurate and more stable than any single model trained on the limited labeled set alone.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Semi-supervised Bagging when labeled data are expensive or scarce — for example, annotations requiring expert review — but large pools of unlabeled observations exist. It is appropriate for both classification and regression on tabular or feature-vector data where semi-supervised learning is applicable. Do not use it when the labeled set is already large enough to train the base learner well (plain bagging suffices), when unlabeled data are drawn from a distribution different from the labeled data (distribution mismatch violates the key assumption), or when pseudo-label quality is persistently low — confidence filtering will reject most unlabeled points and provide little benefit.
Strengths & limitations
- Exploits abundant unlabeled data to improve accuracy without additional labeling cost.
- Inherits bagging's variance reduction: diverse bootstrap samples prevent overfitting to noisy pseudo-labels.
- Model-agnostic framework — any differentiable or non-differentiable base learner can be plugged in.
- Confidence-threshold filtering provides a built-in safeguard against propagating low-quality pseudo-labels.
- Naturally parallelizable: each base learner trains independently, scaling to large datasets.
- Requires that labeled and unlabeled data share the same underlying distribution; distribution mismatch degrades performance.
- Pseudo-label errors can compound across iterations if confidence thresholds are set too low.
- Hyperparameter sensitivity: the confidence threshold, number of base learners, and number of pseudo-labeling rounds all interact and need tuning.
- Provides no explicit feature-importance or coefficient outputs, limiting interpretability for many academic audiences.
Frequently asked
How is Semi-supervised Bagging different from plain self-training?
Plain self-training uses a single model to iteratively pseudo-label and retrain. Semi-supervised Bagging uses an ensemble of models trained on bootstrap samples, so pseudo-label quality is judged by ensemble agreement rather than a single model's confidence, which reduces error propagation.
How should I set the confidence threshold?
A common starting point is to accept only pseudo-labels where the ensemble posterior exceeds 0.8–0.9 for classification. Tune via cross-validation on the labeled set; if too few unlabeled points pass the threshold, lower it slightly — but watch for accuracy degradation.
How many base learners are needed?
The same guidance as standard bagging applies: start with 100 and increase until performance on a held-out labeled validation set stabilises. More learners reduce variance but increase compute.
Can I use this with neural networks as base learners?
Yes, but in practice decision trees or shallow models are more common because they are fast to retrain on each bootstrap sample. Neural networks can serve as base learners if compute allows, though methods like pseudo-label training or mean-teacher are more typical for deep semi-supervised learning.
What if my unlabeled set is much larger than my labeled set?
This is the typical beneficial scenario. Apply aggressive confidence filtering to avoid swamping the bootstrap samples with noisy pseudo-labels, and consider weighting labeled examples more heavily during resampling to preserve their influence.
Sources
- Bennett, K. P., & Demiriz, A. (1999). Semi-supervised support vector machines. Advances in Neural Information Processing Systems, 11. MIT Press. link ↗
- Li, M., & Zhou, Z.-H. (2005). SETRED: Self-training with editing. In Proceedings of the 9th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), LNAI 3518, pp. 611–621. Springer. DOI: 10.1007/11430919_71 ↗
How to cite this page
ScholarGate. (2026, June 3). Semi-supervised Bagging (Bootstrap Aggregating with Unlabeled Data). ScholarGate. https://scholargate.app/en/machine-learning/semi-supervised-bagging
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Gradient BoostingMachine learning↔ compare
- Label PropagationMachine learning↔ compare
- Random ForestMachine learning↔ compare
- Semi-supervised LearningMachine learning↔ compare