Semi-supervised Decision Tree
Semi-supervised Decision Tree Learning · Also known as: SSDT, semi-supervised tree induction, self-training decision tree, label-propagation tree
A Semi-supervised Decision Tree extends standard decision tree induction — such as CART or C4.5 — to exploit unlabeled observations alongside the labeled training set. By iteratively assigning tentative labels to unlabeled data and incorporating them into the growing or splitting process, the algorithm can achieve better accuracy than a fully supervised tree trained on the labeled subset alone, which is especially valuable when labeling is expensive or time-consuming.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Semi-supervised decision trees are appropriate when labeled data are scarce and expensive to obtain but a large pool of unlabeled observations is available — common in medical imaging annotation, text classification, and sensor-based monitoring. They are particularly useful when the overall feature distribution is well captured by the unlabeled pool and when a human-interpretable model (a tree) is required. Avoid this method when the labeled sample is so small that the initial tree is highly unreliable, when unlabeled data are not representative of the test distribution, or when the pseudo-labeling process is expected to propagate systematic errors (confirmation bias). In those cases, purely supervised methods on clean labeled data, or alternative semi-supervised approaches such as label propagation or co-training, may be safer.
Strengths & limitations
- Leverages unlabeled data to improve accuracy when labeled examples are few, reducing annotation cost.
- Retains the interpretability of decision trees: the resulting model is a human-readable set of if-then rules.
- Naturally handles mixed feature types — continuous, categorical, binary — without preprocessing normalization.
- Iterative self-training is straightforward to implement on top of any existing tree learner.
- Can capture nonlinear boundaries and feature interactions that linear semi-supervised models miss.
- Pseudo-label errors from an unreliable initial tree can be amplified across iterations, degrading performance relative to a fully supervised baseline.
- The method assumes that the unlabeled data share the same underlying distribution as the labeled data; distribution shift breaks this assumption.
- Computational cost grows with each iteration as trees are retrained on increasingly large effective datasets.
- Selecting the confidence threshold for pseudo-label inclusion requires careful tuning and cross-validation.
Frequently asked
How is a semi-supervised decision tree different from a regular self-training classifier?
The core principle — using predictions on unlabeled data to augment training — is the same. What distinguishes tree-specific approaches is the option to modify the splitting criterion itself to incorporate unlabeled distribution information, rather than just expanding the labeled set iteratively.
How many labeled examples do I need to start?
There is no universal minimum, but the initial supervised tree must be reliable enough to produce useful pseudo-labels. In practice, at least 20–50 labeled examples per class are often recommended; with fewer, the error propagation risk is high and alternative approaches such as label propagation may work better.
Should I use all unlabeled data or select a subset?
Selecting only high-confidence pseudo-labels — for example those where the leaf probability exceeds a threshold such as 0.85 — generally outperforms using all unlabeled points, especially when the initial tree is imperfect.
Does this replace standard cross-validation?
No. Because pseudo-labels are derived from the model itself, in-sample metrics are optimistic. Evaluation must use a held-out labeled test set or k-fold cross-validation applied strictly to the labeled partition.
Can I combine this with ensemble methods?
Yes. Semi-supervised random forests and co-forest approaches extend the idea to ensembles, which can reduce the variance of pseudo-label errors and typically outperform a single semi-supervised tree.
Sources
- Levin, E. & Shapiro, E. (2000). Learning Decision Trees from Semi-labeled Examples. Proceedings of the ICML Workshop on Attribute-Value and Relational Learning. link ↗
- Zhu, X. & Goldberg, A. B. (2009). Introduction to Semi-Supervised Learning. Morgan & Claypool Publishers. ISBN: 978-1-598-29548-9
How to cite this page
ScholarGate. (2026, June 3). Semi-supervised Decision Tree Learning. ScholarGate. https://scholargate.app/en/machine-learning/semi-supervised-decision-tree
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Decision TreeMachine learning↔ compare
- Gradient BoostingMachine learning↔ compare
- Label PropagationMachine learning↔ compare
- Random ForestMachine learning↔ compare