Weakly Supervised Question Answering
Also known as: WS-QA, distantly supervised QA, noisy-label question answering, indirect supervision QA
Weakly supervised question answering (WS-QA) trains neural reading-comprehension models using indirect or automatically derived answer labels rather than expensive human-annotated span annotations. By exploiting distant supervision, heuristic labeling, or answer-presence signals, WS-QA makes QA feasible in domains and languages where full annotation is impractical.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use weakly supervised QA when labeled answer-span annotations are unavailable or prohibitively expensive but some indirect signal exists, such as known answer strings, knowledge-base facts, or binary answer-present flags. It is well suited to low-resource domains — medical, legal, scientific — where expert annotation is scarce and to new languages where large supervised QA datasets do not exist. Do not use it when fully annotated span-level data is available, as standard fine-tuning will outperform weak supervision. Avoid it when the weak labels are so noisy that no signal remains, or when exact span precision is legally or clinically critical without post-hoc human verification.
Strengths & limitations
- Enables QA model training without expensive span-level annotation by exploiting freely available weak signals.
- Scales to low-resource domains and languages where fully labeled QA datasets are absent.
- Compatible with powerful pre-trained language models (BERT, RoBERTa, T5) as the reader backbone.
- Iterative self-training loops can progressively improve accuracy beyond the initial weak-label quality.
- Distant supervision from knowledge bases or answer strings is cheap to generate at scale.
- Answer quality is bounded by the quality and coverage of the weak signals; highly noisy signals lead to poorly calibrated models.
- Label noise can cause the model to learn spurious correlations between surface patterns and answers.
- Evaluation is harder because retrieved weak labels may inflate apparent performance if the test set shares the same bias.
- Iterative self-training risks reinforcing early model errors, creating confirmation loops.
Frequently asked
What counts as weak supervision in QA?
Any label that is cheaper than a human-annotated answer span: knowing only that a passage contains the answer string (distant supervision), a binary yes/no flag for answer presence, or automatically extracted answer candidates from a knowledge base.
How does hard EM help with noisy labels?
Hard expectation-maximization treats the correct passage or span as a latent variable. At each iteration it picks the highest-scoring candidate as the pseudo-ground-truth and updates the model, iteratively sharpening localization despite the absence of exact annotations.
Can I use weakly supervised QA with GPT-style models?
Yes. Large language models can be fine-tuned or prompted with weak supervision signals, and self-training loops using model-generated answers as pseudo-labels are common. The pre-trained knowledge often compensates for label noise.
When should I prefer semi-supervised over weakly supervised QA?
Choose semi-supervised QA when you have a small but clean set of span-annotated examples alongside a large unlabeled corpus. Choose weakly supervised QA when even a small clean set is unavailable but indirect answer signals exist.
How do I evaluate a weakly supervised QA model fairly?
Use a held-out test set with human-verified answer spans that was not derived from the same weak-labeling pipeline as training. Standard metrics are Exact Match and F1 over the answer string.
Sources
- Clark, C., & Gardner, M. (2018). Simple and Effective Multi-Paragraph Reading Comprehension. In Proceedings of ACL 2018, pp. 845–855. Association for Computational Linguistics. link ↗
- Min, S., Chen, D., Hajishirzi, H., & Zettlemoyer, L. (2019). A Discrete Hard EM Approach for Weakly Supervised Question Answering. In Proceedings of EMNLP-IJCNLP 2019, pp. 2083–2093. Association for Computational Linguistics. link ↗
How to cite this page
ScholarGate. (2026, June 3). Weakly Supervised Question Answering. ScholarGate. https://scholargate.app/en/deep-learning/weakly-supervised-question-answering
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- BERT-based ClassificationDeep learning↔ compare
- Domain-adaptive Question AnsweringDeep learning↔ compare
- Fine-Tuned Question AnsweringDeep learning↔ compare
- Semi-supervised Question AnsweringDeep learning↔ compare