Semi-supervised Question Answering
Semi-supervised Question Answering (Self-Training and Consistency-Based NLP) · Also known as: Semi-supervised QA, Self-training for QA, Pseudo-labeled Question Answering, SSL-QA
Semi-supervised question answering (QA) trains a model on a small labeled set of question-answer pairs, then generates pseudo-labels on a large unlabeled corpus and retrains iteratively. This self-training loop dramatically increases effective training data without the cost of full manual annotation, achieving strong performance on reading comprehension, open-domain QA, and machine reading tasks.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use semi-supervised QA when you have a small labeled QA dataset (dozens to a few hundred annotated triples) but access to a large unlabeled domain corpus — common in specialized domains such as biomedical, legal, or technical literature where annotation is costly. It is particularly valuable for low-resource languages or niche domains where pre-trained QA models do not transfer well. Do not use it when labeled data is already abundant (hundreds of thousands of examples) because the marginal gain is small relative to the added complexity. Avoid it when unlabeled domain text is not available or is very short, and when the base model confidence is consistently low — in that case, noise from poor pseudo-labels can hurt rather than help.
Strengths & limitations
- Dramatically increases effective training size without proportional annotation cost.
- Exploits large unlabeled corpora that are freely available in most domains.
- Confidence filtering limits noise injection from low-quality pseudo-labels.
- Compatible with any pre-trained transformer backbone (BERT, RoBERTa, ELECTRA, etc.).
- Iterative self-training can progressively improve both label quality and model accuracy.
- Applicable to both extractive and generative question-answering settings.
- Quality of pseudo-labels is bounded by the seed model; a weak seed produces noisy labels that can degrade performance.
- Requires access to a large, domain-relevant unlabeled corpus — not always available.
- Multiple retraining rounds are computationally expensive, especially with large transformer models.
- Confidence threshold is a sensitive hyperparameter: too high filters useful data; too low admits noise.
- Model can develop confirmation bias, becoming increasingly confident in its own errors over rounds.
Frequently asked
How many labeled examples do I need to start?
Practical studies show useful self-training gains starting from a few hundred labeled QA triples, provided the seed model is a pre-trained transformer. Fewer than 50 examples typically yield a seed model too weak to generate reliable pseudo-labels.
What confidence threshold should I use?
There is no universal value — tune it on your development set. Common practice is to start at a high threshold (e.g., top-30% confidence) and relax it across rounds as the model improves. Monitor dev-set F1 at each round rather than using a fixed threshold throughout.
How many self-training rounds are sufficient?
Most published work reports gains plateau within 2–4 rounds. Use early stopping based on dev-set EM or F1; if performance does not improve for one or two consecutive rounds, stop retraining.
Can I combine this with data augmentation?
Yes — question generation models (e.g., T5 fine-tuned on SQuAD) can generate additional (question, passage) pairs from unlabeled text, complementing pseudo-labeled self-training and further expanding the effective training set.
Is this the same as transfer learning or fine-tuning?
No. Transfer learning reuses representations from a pre-trained model on a labeled target dataset. Semi-supervised QA additionally exploits unlabeled target-domain text through iterative pseudo-labeling, making it a distinct strategy suited for low-label regimes.
Sources
- Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020). ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. In Proceedings of ICLR 2020. link ↗
- Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019). XLNet: Generalized Autoregressive Pretraining for Language Understanding. In Advances in Neural Information Processing Systems (NeurIPS 2019). link ↗
How to cite this page
ScholarGate. (2026, June 3). Semi-supervised Question Answering (Self-Training and Consistency-Based NLP). ScholarGate. https://scholargate.app/en/deep-learning/semi-supervised-question-answering
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- BERT-based ClassificationDeep learning↔ compare
- Fine-Tuned Question AnsweringDeep learning↔ compare
- Self-supervised Question AnsweringDeep learning↔ compare
- Semi-supervised BERT-based ClassificationDeep learning↔ compare
- Semi-supervised TransformerDeep learning↔ compare
- Weakly supervised question answeringDeep learning↔ compare