Semi-supervised Naive Bayes
Semi-supervised Naive Bayes (EM-augmented Generative Classifier) · Also known as: SSL Naive Bayes, EM-Naive Bayes, semi-supervised generative classifier, Nigam et al. text classifier
Semi-supervised Naive Bayes extends the classic Naive Bayes generative model to exploit large pools of unlabeled data alongside a small labeled set. Using Expectation-Maximization, it iteratively infers soft class assignments for unlabeled examples and re-estimates class and feature parameters, yielding substantially better classifiers when labeled examples are scarce.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Ideal when labeled examples number in the tens to low hundreds but large amounts of unlabeled data from the same distribution are available — a common situation in text classification, spam filtering, and biomedical document categorization. Features should be conditionally independent given class (or approximately so), making bag-of-words and count-based representations natural fits. Avoid when the unlabeled data comes from a different distribution than the labeled set (covariate shift), as the EM updates can hurt rather than help; the lambda weighting parameter should be tuned in such cases. Also avoid when the conditional independence assumption is severely violated and a discriminative model would be strongly preferred regardless of label scarcity.
Strengths & limitations
- Efficiently leverages unlabeled data that would otherwise be discarded, often dramatically reducing the labeled data needed.
- Computationally fast: each EM iteration is linear in data size, making it suitable for large text corpora.
- Principled probabilistic framework with well-understood convergence guarantees (to a local optimum of the likelihood).
- Produces calibrated posterior probabilities, enabling downstream ranking and thresholding.
- Simple to implement and extend — lambda weighting provides a practical knob to control the unlabeled data influence.
- Inherits the Naive Bayes conditional-independence assumption; correlated features degrade parameter estimates.
- EM converges to a local optimum that depends on initialization; multiple random restarts may be needed.
- Unlabeled data from a shifted distribution can degrade accuracy (negative transfer) unless carefully down-weighted.
- Less competitive than discriminative semi-supervised methods (e.g., transductive SVM, label propagation) on small feature sets with strong feature correlations.
Frequently asked
How many labeled examples do I need to get a good starting model?
As few as 10–50 per class can suffice for text with many features, provided Laplace smoothing is applied. More labeled examples always help, but the semi-supervised approach specifically shines when you have fewer than a few hundred labeled instances.
What is the lambda parameter and how do I tune it?
Lambda controls how much weight the unlabeled examples receive relative to labeled ones during the M-step (lambda=1 treats them equally; lambda=0 ignores them). Tune it via cross-validation on the labeled set, starting with values in {0.1, 0.3, 0.5, 1.0}. Reduce lambda when you suspect distribution mismatch.
Will EM always improve on a labeled-only Naive Bayes?
No. If the unlabeled data come from a different distribution, EM can move parameters in the wrong direction and increase error. Always benchmark against the labeled-only baseline before deploying the semi-supervised model.
How does this differ from self-training with Naive Bayes?
Self-training assigns hard labels to unlabeled examples above a confidence threshold, then retrains. Semi-supervised Naive Bayes via EM assigns soft fractional labels to all unlabeled examples at every iteration, which is statistically more principled and avoids the cascade of confident errors that can occur in self-training.
Is this suitable for non-text tabular data?
Yes. For continuous features replace the multinomial likelihood with Gaussian (or other parametric) likelihoods per class. The EM loop structure is identical; only the M-step parameter formulas change.
Sources
- Nigam, K., McCallum, A. K., Thrun, S., & Mitchell, T. (2000). Text Classification from Labeled and Unlabeled Documents using EM. Machine Learning, 39(2–3), 103–134. DOI: 10.1023/A:1007692713085 ↗
- Chapelle, O., Scholkopf, B., & Zien, A. (Eds.). (2006). Semi-Supervised Learning. MIT Press. ISBN: 978-0-262-03358-9
How to cite this page
ScholarGate. (2026, June 3). Semi-supervised Naive Bayes (EM-augmented Generative Classifier). ScholarGate. https://scholargate.app/en/machine-learning/semi-supervised-naive-bayes
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Logistic RegressionResearch Statistics↔ compare
- Naive BayesMachine learning↔ compare
- Semi-supervised LearningMachine learning↔ compare
- Semi-supervised Support Vector MachineMachine learning↔ compare