Bayesian Semi-supervised Learning
Bayesian Semi-supervised Learning (Probabilistic Inference with Labeled and Unlabeled Data) · Also known as: Bayesian SSL, probabilistic semi-supervised learning, generative semi-supervised model, Bayesian transductive learning
Bayesian semi-supervised learning is a probabilistic framework that uses both a small labeled dataset and a larger pool of unlabeled observations to infer model parameters and make predictions. By treating missing labels as latent variables and placing priors over parameters, it naturally quantifies uncertainty while leveraging unlabeled data to improve generalization.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Bayesian semi-supervised learning when labeled examples are expensive or scarce but large quantities of unlabeled data are available, and when calibrated uncertainty estimates matter alongside accuracy — for example in medical diagnosis, active scientific data collection, or safety-critical prediction. It is well suited to settings where the data cluster smoothly and the generative model assumptions are approximately correct. Avoid it when labeled data are plentiful (fully supervised Bayesian models suffice), when the generative model is badly misspecified, or when computational cost is a hard constraint, as posterior inference is substantially more expensive than discriminative training.
Strengths & limitations
- Formally incorporates unlabeled data through marginalization, improving generalization when labels are scarce.
- Produces calibrated predictive distributions and explicit uncertainty estimates, not just point predictions.
- Prior distributions allow principled injection of domain knowledge or regularization.
- Unified probabilistic framework supports downstream active learning and decision-theoretic choices.
- Model comparison and selection can be performed via marginal likelihood (Bayesian model evidence).
- Posterior inference is often intractable and requires approximate methods such as variational EM or MCMC, adding implementation complexity.
- Performance depends heavily on how well the generative model matches the true data distribution; misspecification can degrade accuracy below a purely discriminative baseline.
- Computational cost scales poorly with dataset size compared to simple discriminative semi-supervised methods.
- Selecting and validating priors is non-trivial and can introduce hidden researcher degrees of freedom.
Frequently asked
How does Bayesian SSL differ from standard semi-supervised learning?
Standard semi-supervised methods such as label propagation or self-training produce point estimates. Bayesian SSL maintains a full posterior over parameters, delivering calibrated predictive uncertainty and supporting principled model comparison via marginal likelihood.
Can unlabeled data ever hurt performance in this framework?
Yes. If the generative model is misspecified — for instance if the marginal distribution of features in unlabeled data differs from the labeled pool — incorporating unlabeled data can pull the posterior toward wrong parameter values, worsening accuracy compared to a purely supervised baseline.
What inference method should I use?
Variational EM (expectation-maximization with a variational lower bound) is the most common choice for large datasets due to its scalability. MCMC offers more accurate posteriors but is feasible only for smaller models. For deep generative models, amortized variational inference via the VAE framework is standard.
How much labeled data do I need?
The framework is designed for settings with very few labels — sometimes as few as one or two per class. Performance improvements from unlabeled data are most pronounced when labeled examples number in the tens to low hundreds; beyond a few thousand labels, purely discriminative models often match or exceed it.
Is Bayesian SSL related to Gaussian process semi-supervised learning?
Yes. Gaussian process classifiers and regressors are a specific instantiation of Bayesian SSL in which the prior is a GP over functions. They are especially popular for small datasets requiring smooth, well-calibrated predictions.
Sources
- Chapelle, O., Scholkopf, B., & Zien, A. (Eds.). (2006). Semi-Supervised Learning. MIT Press. ISBN: 978-0-262-03358-9
- Zhu, X., Ghahramani, Z., & Lafferty, J. (2003). Semi-supervised learning using Gaussian fields and harmonic functions. Proceedings of the 20th International Conference on Machine Learning (ICML), 912–919. link ↗
How to cite this page
ScholarGate. (2026, June 3). Bayesian Semi-supervised Learning (Probabilistic Inference with Labeled and Unlabeled Data). ScholarGate. https://scholargate.app/en/machine-learning/bayesian-semi-supervised-learning
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Active LearningMachine learning↔ compare
- Bayesian Gaussian Mixture ModelMachine learning↔ compare
- Few-shot LearningMachine learning↔ compare
- Gaussian ProcessMachine learning↔ compare
- Semi-supervised LearningMachine learning↔ compare
- Transfer LearningMachine learning↔ compare