Semi-supervised Doc2Vec
Semi-supervised Paragraph Vector (Semi-supervised Doc2Vec) · Also known as: Semi-supervised Paragraph Vector, SS-Doc2Vec, Label-guided PV-DBOW, Semi-supervised PV-DM
Semi-supervised Doc2Vec extends the Paragraph Vector framework of Le and Mikolov (2014) by training dense document embeddings on both labeled and unlabeled corpora simultaneously, using available class labels as an auxiliary signal to steer the representation toward task-relevant structure while still exploiting the full unlabeled collection for generalization.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Best suited for text classification, sentiment analysis, or topic labeling tasks where labeled examples are scarce (hundreds to low thousands) but a large unlabeled corpus exists (tens of thousands or more). Particularly valuable in domain-specific settings — clinical notes, legal documents, scientific abstracts — where pre-trained transformer models have weak priors. Avoid when the unlabeled corpus is small (embeddings will underfit), when real-time document vector inference is required at scale (inference by gradient descent is slow), or when a large labeled dataset is available — in that case, supervised fine-tuning of a transformer is likely superior.
Strengths & limitations
- Leverages unlabeled data to learn richer document representations than a purely supervised model can achieve from few labels.
- Produces a single fixed-length vector per document, making downstream classification simple and fast.
- Architecture is lightweight compared to transformer-based models and can run on CPU-only hardware.
- The label-injection weight is a single tunable hyperparameter that lets practitioners trade off supervised vs. unsupervised signal.
- Works on any language without requiring a language-specific pretrained model.
- Inference for unseen documents requires an additional gradient-descent pass per document, which is computationally expensive at large scale.
- The method is sensitive to the labeled/unlabeled ratio and the weight assigned to the supervised auxiliary loss.
- In high-resource settings (large labeled corpora), fine-tuned transformer models (BERT, RoBERTa) consistently outperform Doc2Vec-based approaches.
- The model does not capture long-range syntactic or semantic dependencies as effectively as attention-based architectures.
Frequently asked
How is semi-supervised Doc2Vec different from plain Doc2Vec?
Plain Doc2Vec is entirely unsupervised: it learns document vectors purely from word co-occurrence without using any class labels. The semi-supervised variant adds a class-conditional loss for the labeled subset, bending the embedding geometry toward discriminative structure while still exploiting all unlabeled documents for coverage.
When should I prefer semi-supervised Doc2Vec over BERT fine-tuning?
When you have very few labeled examples (< 500) and a large unlabeled corpus in a narrow domain where general-purpose pretrained transformers have weak coverage — e.g., proprietary clinical records or highly technical legal text — semi-supervised Doc2Vec can match or exceed BERT fine-tuning at a fraction of the compute cost.
How do I set the auxiliary label-loss weight?
Start with a weight of 0.1–0.5 relative to the unsupervised reconstruction loss and tune it on a held-out labeled validation set. A weight above 1.0 risks collapsing the unsupervised signal; a weight below 0.01 effectively reduces to plain Doc2Vec.
Can I use semi-supervised Doc2Vec for multi-label classification?
Yes. Replace the softmax cross-entropy auxiliary loss with a sigmoid binary cross-entropy loss applied independently to each label. The rest of the training pipeline is identical.
Does the unlabeled corpus need to be in the same domain as the labeled data?
Not strictly, but the closer the domains are, the more the unlabeled data benefits the task-relevant embedding. Out-of-domain unlabeled text dilutes the signal and may hurt performance if it dominates the corpus.
Sources
How to cite this page
ScholarGate. (2026, June 3). Semi-supervised Paragraph Vector (Semi-supervised Doc2Vec). ScholarGate. https://scholargate.app/en/deep-learning/semi-supervised-doc2vec
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
Compare side by side →