Semi-supervised Doc2Vec
Also known as: Semi-supervised Paragraph Vector, SS-Doc2Vec, Label-guided PV-DBOW, Semi-supervised PV-DM
Semi-supervised Doc2Vec extends the Paragraph Vector framework of Le and Mikolov (2014) by training dense document embeddings on both labeled and unlabeled corpora simultaneously, using available class labels as an auxiliary signal to steer the representation toward task-relevant structure while still exploiting the full unlabeled collection for generalization.
Key highlights
- Leverages unlabeled data to learn richer document representations than a purely supervised model can achieve from few labels.
- Produces a single fixed-length vector per document, making downstream classification simple and fast.
- Architecture is lightweight compared to transformer-based models and can run on CPU-only hardware.
- The label-injection weight is a single tunable hyperparameter that lets practitioners trade off supervised vs. unsupervised signal.
- Works on any language without requiring a language-specific pretrained model.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Best suited for text classification, sentiment analysis, or topic labeling tasks where labeled examples are scarce (hundreds to low thousands) but a large unlabeled corpus exists (tens of thousands or more). Particularly valuable in domain-specific settings — clinical notes, legal documents, scientific abstracts — where pre-trained transformer models have weak priors. Avoid when the unlabeled corpus is small (embeddings will underfit), when real-time document vector inference is required at scale (inference by gradient descent is slow), or when a large labeled dataset is available — in that case, supervised fine-tuning of a transformer is likely superior.
Strengths & limitations
- Leverages unlabeled data to learn richer document representations than a purely supervised model can achieve from few labels.
- Produces a single fixed-length vector per document, making downstream classification simple and fast.
- Architecture is lightweight compared to transformer-based models and can run on CPU-only hardware.
- The label-injection weight is a single tunable hyperparameter that lets practitioners trade off supervised vs. unsupervised signal.
- Works on any language without requiring a language-specific pretrained model.
- Inference for unseen documents requires an additional gradient-descent pass per document, which is computationally expensive at large scale.
- The method is sensitive to the labeled/unlabeled ratio and the weight assigned to the supervised auxiliary loss.
- In high-resource settings (large labeled corpora), fine-tuned transformer models (BERT, RoBERTa) consistently outperform Doc2Vec-based approaches.
- The model does not capture long-range syntactic or semantic dependencies as effectively as attention-based architectures.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How is semi-supervised Doc2Vec different from plain Doc2Vec?
Plain Doc2Vec is entirely unsupervised: it learns document vectors purely from word co-occurrence without using any class labels. The semi-supervised variant adds a class-conditional loss for the labeled subset, bending the embedding geometry toward discriminative structure while still exploiting all unlabeled documents for coverage.
When should I prefer semi-supervised Doc2Vec over BERT fine-tuning?
When you have very few labeled examples (< 500) and a large unlabeled corpus in a narrow domain where general-purpose pretrained transformers have weak coverage — e.g., proprietary clinical records or highly technical legal text — semi-supervised Doc2Vec can match or exceed BERT fine-tuning at a fraction of the compute cost.
How do I set the auxiliary label-loss weight?
Start with a weight of 0.1–0.5 relative to the unsupervised reconstruction loss and tune it on a held-out labeled validation set. A weight above 1.0 risks collapsing the unsupervised signal; a weight below 0.01 effectively reduces to plain Doc2Vec.
Can I use semi-supervised Doc2Vec for multi-label classification?
Yes. Replace the softmax cross-entropy auxiliary loss with a sigmoid binary cross-entropy loss applied independently to each label. The rest of the training pipeline is identical.
Does the unlabeled corpus need to be in the same domain as the labeled data?
Not strictly, but the closer the domains are, the more the unlabeled data benefits the task-relevant embedding. Out-of-domain unlabeled text dilutes the signal and may hurt performance if it dominates the corpus.
Sources
- 1.Le, Q. V., & Mikolov, T. (2014). Distributed Representations of Sentences and Documents. Proceedings of the 31st International Conference on Machine Learning (ICML 2014), PMLR 32(2), 1188–1196.
- 2.Word2vec. Wikipedia.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Semi-supervised Doc2Vec. ScholarGate. https://scholargate.app/deep-learning/semi-supervised-doc2vec