Semi-supervised Logistic Regression
Semi-supervised Logistic Regression (Self-training and EM-based variants) · Also known as: SSL logistic regression, semi-supervised LR, EM logistic regression, self-training logistic classifier
Semi-supervised logistic regression extends the standard logistic classifier by incorporating unlabeled data during training. Using self-training, expectation-maximization, or label-propagation wrappers, it iteratively assigns soft labels to unlabeled examples and refines model parameters, improving generalization when labeled data are scarce relative to the full dataset.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Choose semi-supervised logistic regression when labeled examples are few — typically when the labeled fraction is under 20% of the total available data — but a large pool of unlabeled data from the same distribution exists and annotation is expensive. It is well-suited to text classification, clinical outcome prediction, and survey research where outcomes are partially observed. Do not use it when the unlabeled data come from a different distribution than the labeled set (covariate shift), when labels are missing not at random and missingness is informative, or when the labeled set is already large enough that a standard logistic regression achieves good performance — in those cases, the pseudo-label noise adds more harm than benefit.
Strengths & limitations
- Exploits large pools of unlabeled data to improve a logistic classifier without additional labeling cost.
- Preserves the interpretability of logistic regression coefficients and odds ratios.
- Self-training wrapper is model-agnostic and easy to add to any existing logistic regression pipeline.
- EM variant provides a principled probabilistic framework with a formal convergence guarantee.
- Regularization (L1/L2) can be combined straightforwardly to control overfitting on pseudo-labeled data.
- If initial labeled data are too few or unrepresentative, early pseudo-labels are noisy and errors compound across iterations.
- Assumes that the unlabeled data are drawn from the same distribution as the labeled data — violated in many real-world collection scenarios.
- Convergence to a local optimum is not guaranteed to improve over the supervised baseline.
- Confidence thresholds in self-training require tuning and interact with class imbalance in non-obvious ways.
- Interpreting coefficients becomes harder because the effective training set includes algorithmically generated labels.
Frequently asked
How is semi-supervised logistic regression different from standard logistic regression?
Standard logistic regression uses only labeled examples. The semi-supervised variant iteratively assigns tentative labels to unlabeled data and includes them in retraining, so the decision boundary is shaped by both confirmed labels and the structure of the larger unlabeled dataset.
What confidence threshold should I use for self-training?
Common starting points are 0.85–0.95 for binary classification. The right threshold depends on class balance and label quality. Tune it on the labeled validation set and monitor the fraction of unlabeled examples added per iteration to avoid a flood of uncertain pseudo-labels.
Can I still interpret the coefficients as odds ratios?
Yes, but with caution. The estimated coefficients retain the log-odds interpretation, but their confidence intervals are underestimated because pseudo-labeled data are treated as real observations. Inference about significance should rely only on the truly labeled portion.
What if my unlabeled data come from a different time period or source?
Distribution shift between labeled and unlabeled sets is one of the main failure modes. If the unlabeled data come from a different population or period, pseudo-labels may reinforce systematic errors. Check for covariate shift before proceeding.
Is there a ready-made implementation?
Yes. scikit-learn's SelfTrainingClassifier wraps any probabilistic classifier, including LogisticRegression, for self-training. The EM variant requires manual implementation or dedicated SSL libraries such as SKSSL or LASSL.
Sources
- Nigam, K., McCallum, A., Thrun, S., & Mitchell, T. (2000). Text classification from labeled and unlabeled documents using EM. Machine Learning, 39, 103–134. DOI: 10.1023/a:1007692713085 ↗
- Chapelle, O., Scholkopf, B., & Zien, A. (Eds.) (2006). Semi-supervised Learning. MIT Press. ISBN: 978-0-262-03358-9
How to cite this page
ScholarGate. (2026, June 3). Semi-supervised Logistic Regression (Self-training and EM-based variants). ScholarGate. https://scholargate.app/en/machine-learning/semi-supervised-logistic-regression
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Label PropagationMachine learning↔ compare
- Logistic regression (ML)Machine learning↔ compare
- Self-supervised Logistic RegressionMachine learning↔ compare
- Semi-supervised LearningMachine learning↔ compare
- Semi-supervised Naive BayesMachine learning↔ compare