Semi-supervised XGBoost
Also known as: SS-XGBoost, semi-supervised gradient boosting, pseudo-label XGBoost, label-propagation XGBoost
Semi-supervised XGBoost extends the XGBoost gradient boosting framework to settings where only a fraction of training examples carry labels. By iteratively generating pseudo-labels for unlabeled data and retraining on the expanded set, the method extracts signal from unlabeled observations, improving generalization when labeled data are scarce.
Key highlights
- Exploits unlabeled data to improve generalisation without requiring additional annotation effort.
- Inherits XGBoost's strong performance on tabular data: handles mixed feature types, missing values, and nonlinear interactions natively.
- Confidence-threshold filtering controls pseudo-label noise, making the approach more robust than naive self-training.
- Plug-and-play: compatible with any XGBoost objective and hyperparameter configuration; pseudo-labeling can wrap existing pipelines.
- Iterative refinement progressively increases the quality of pseudo-labels as the model improves.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use semi-supervised XGBoost when labeled examples are fewer than needed for a reliable supervised model yet a substantial pool of unlabeled examples with the same feature space is available — common in medical informatics, fraud detection, and industrial quality control. It is well-suited to tabular data with structured features, heterogeneous types, and moderate dimensionality. Avoid it when labeled data are already plentiful and the overhead of pseudo-labeling adds complexity without accuracy gains, when the labeled and unlabeled distributions differ substantially (covariate shift), or when the initial supervised model is too weak to generate trustworthy pseudo-labels (very small labeled set, below roughly 50 examples).
Strengths & limitations
- Exploits unlabeled data to improve generalisation without requiring additional annotation effort.
- Inherits XGBoost's strong performance on tabular data: handles mixed feature types, missing values, and nonlinear interactions natively.
- Confidence-threshold filtering controls pseudo-label noise, making the approach more robust than naive self-training.
- Plug-and-play: compatible with any XGBoost objective and hyperparameter configuration; pseudo-labeling can wrap existing pipelines.
- Iterative refinement progressively increases the quality of pseudo-labels as the model improves.
- Performance degrades sharply if the initial labeled set is too small to produce reliable pseudo-labels — error accumulates across iterations.
- Covariate shift between labeled and unlabeled data can cause pseudo-labels to reflect the labeled distribution rather than the true unlabeled one.
- Threshold selection is heuristic; an ill-chosen confidence cutoff either admits too many noisy labels or rejects useful unlabeled examples.
- Interpretability remains limited — the semi-supervised wrapper adds a layer of complexity on top of XGBoost's already opaque ensemble.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How many labeled examples are needed to start?
There is no hard minimum, but fewer than roughly 50 labeled examples typically yield pseudo-labels too noisy to be useful. Performance gains from semi-supervised training become most pronounced when labeled examples number in the hundreds while unlabeled examples number in the thousands or more.
How should the confidence threshold be set?
A common default for binary classification is to accept pseudo-labels only when predicted probability exceeds 0.85–0.90. The threshold should be tuned on a held-out labeled validation set: tighten it if pseudo-label accuracy is low, relax it if too few unlabeled points are being incorporated.
Does semi-supervised XGBoost always outperform supervised XGBoost?
Not always. When the labeled set is already representative and large, the semi-supervised extension adds complexity without meaningful accuracy gains. It also underperforms when the unlabeled distribution is shifted relative to the labeled one.
Is this the same as transductive XGBoost or label propagation?
Related but distinct. Pseudo-label semi-supervised XGBoost is an inductive method that produces a trained model applicable to new test points. Label propagation is a graph-based method that assigns labels only to the seen unlabeled set and does not generalise as directly to new data.
How do I prevent error accumulation across iterations?
Accept only high-confidence pseudo-labels, weight them below genuine labels, limit the number of iterations, and monitor held-out labeled-set accuracy after each round — stopping when it plateaus or begins to decline.
Sources
- 1.Chen, T. & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785–794.
- 2.Chapelle, O., Scholkopf, B. & Zien, A. (Eds.) (2006). Semi-Supervised Learning. MIT Press.ISBN 978-0-262-03358-9
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Semi-supervised XGBoost. ScholarGate. https://scholargate.app/machine-learning/semi-supervised-xgboost