Semi-supervised Question Answering
Also known as: Semi-supervised QA, Self-training for QA, Pseudo-labeled Question Answering, SSL-QA
Semi-supervised question answering (QA) trains a model on a small labeled set of question-answer pairs, then generates pseudo-labels on a large unlabeled corpus and retrains iteratively. This self-training loop dramatically increases effective training data without the cost of full manual annotation, achieving strong performance on reading comprehension, open-domain QA, and machine reading tasks.
Key highlights
- Dramatically increases effective training size without proportional annotation cost.
- Exploits large unlabeled corpora that are freely available in most domains.
- Confidence filtering limits noise injection from low-quality pseudo-labels.
- Compatible with any pre-trained transformer backbone (BERT, RoBERTa, ELECTRA, etc.).
- Iterative self-training can progressively improve both label quality and model accuracy.
- Applicable to both extractive and generative question-answering settings.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use semi-supervised QA when you have a small labeled QA dataset (dozens to a few hundred annotated triples) but access to a large unlabeled domain corpus — common in specialized domains such as biomedical, legal, or technical literature where annotation is costly. It is particularly valuable for low-resource languages or niche domains where pre-trained QA models do not transfer well. Do not use it when labeled data is already abundant (hundreds of thousands of examples) because the marginal gain is small relative to the added complexity. Avoid it when unlabeled domain text is not available or is very short, and when the base model confidence is consistently low — in that case, noise from poor pseudo-labels can hurt rather than help.
Strengths & limitations
- Dramatically increases effective training size without proportional annotation cost.
- Exploits large unlabeled corpora that are freely available in most domains.
- Confidence filtering limits noise injection from low-quality pseudo-labels.
- Compatible with any pre-trained transformer backbone (BERT, RoBERTa, ELECTRA, etc.).
- Iterative self-training can progressively improve both label quality and model accuracy.
- Applicable to both extractive and generative question-answering settings.
- Quality of pseudo-labels is bounded by the seed model; a weak seed produces noisy labels that can degrade performance.
- Requires access to a large, domain-relevant unlabeled corpus — not always available.
- Multiple retraining rounds are computationally expensive, especially with large transformer models.
- Confidence threshold is a sensitive hyperparameter: too high filters useful data; too low admits noise.
- Model can develop confirmation bias, becoming increasingly confident in its own errors over rounds.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How many labeled examples do I need to start?
Practical studies show useful self-training gains starting from a few hundred labeled QA triples, provided the seed model is a pre-trained transformer. Fewer than 50 examples typically yield a seed model too weak to generate reliable pseudo-labels.
What confidence threshold should I use?
There is no universal value — tune it on your development set. Common practice is to start at a high threshold (e.g., top-30% confidence) and relax it across rounds as the model improves. Monitor dev-set F1 at each round rather than using a fixed threshold throughout.
How many self-training rounds are sufficient?
Most published work reports gains plateau within 2–4 rounds. Use early stopping based on dev-set EM or F1; if performance does not improve for one or two consecutive rounds, stop retraining.
Can I combine this with data augmentation?
Yes — question generation models (e.g., T5 fine-tuned on SQuAD) can generate additional (question, passage) pairs from unlabeled text, complementing pseudo-labeled self-training and further expanding the effective training set.
Is this the same as transfer learning or fine-tuning?
No. Transfer learning reuses representations from a pre-trained model on a labeled target dataset. Semi-supervised QA additionally exploits unlabeled target-domain text through iterative pseudo-labeling, making it a distinct strategy suited for low-label regimes.
Sources
- 1.Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020). ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. In Proceedings of ICLR 2020.
- 2.Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019). XLNet: Generalized Autoregressive Pretraining for Language Understanding. In Advances in Neural Information Processing Systems (NeurIPS 2019).
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Semi-supervised Question Answering. ScholarGate. https://scholargate.app/deep-learning/semi-supervised-question-answering