Weakly Supervised Question Answering
Also known as: WS-QA, distantly supervised QA, noisy-label question answering, indirect supervision QA
Weakly supervised question answering (WS-QA) trains neural reading-comprehension models using indirect or automatically derived answer labels rather than expensive human-annotated span annotations. By exploiting distant supervision, heuristic labeling, or answer-presence signals, WS-QA makes QA feasible in domains and languages where full annotation is impractical.
Key highlights
- Enables QA model training without expensive span-level annotation by exploiting freely available weak signals.
- Scales to low-resource domains and languages where fully labeled QA datasets are absent.
- Compatible with powerful pre-trained language models (BERT, RoBERTa, T5) as the reader backbone.
- Iterative self-training loops can progressively improve accuracy beyond the initial weak-label quality.
- Distant supervision from knowledge bases or answer strings is cheap to generate at scale.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use weakly supervised QA when labeled answer-span annotations are unavailable or prohibitively expensive but some indirect signal exists, such as known answer strings, knowledge-base facts, or binary answer-present flags. It is well suited to low-resource domains — medical, legal, scientific — where expert annotation is scarce and to new languages where large supervised QA datasets do not exist. Do not use it when fully annotated span-level data is available, as standard fine-tuning will outperform weak supervision. Avoid it when the weak labels are so noisy that no signal remains, or when exact span precision is legally or clinically critical without post-hoc human verification.
Strengths & limitations
- Enables QA model training without expensive span-level annotation by exploiting freely available weak signals.
- Scales to low-resource domains and languages where fully labeled QA datasets are absent.
- Compatible with powerful pre-trained language models (BERT, RoBERTa, T5) as the reader backbone.
- Iterative self-training loops can progressively improve accuracy beyond the initial weak-label quality.
- Distant supervision from knowledge bases or answer strings is cheap to generate at scale.
- Answer quality is bounded by the quality and coverage of the weak signals; highly noisy signals lead to poorly calibrated models.
- Label noise can cause the model to learn spurious correlations between surface patterns and answers.
- Evaluation is harder because retrieved weak labels may inflate apparent performance if the test set shares the same bias.
- Iterative self-training risks reinforcing early model errors, creating confirmation loops.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What counts as weak supervision in QA?
Any label that is cheaper than a human-annotated answer span: knowing only that a passage contains the answer string (distant supervision), a binary yes/no flag for answer presence, or automatically extracted answer candidates from a knowledge base.
How does hard EM help with noisy labels?
Hard expectation-maximization treats the correct passage or span as a latent variable. At each iteration it picks the highest-scoring candidate as the pseudo-ground-truth and updates the model, iteratively sharpening localization despite the absence of exact annotations.
Can I use weakly supervised QA with GPT-style models?
Yes. Large language models can be fine-tuned or prompted with weak supervision signals, and self-training loops using model-generated answers as pseudo-labels are common. The pre-trained knowledge often compensates for label noise.
When should I prefer semi-supervised over weakly supervised QA?
Choose semi-supervised QA when you have a small but clean set of span-annotated examples alongside a large unlabeled corpus. Choose weakly supervised QA when even a small clean set is unavailable but indirect answer signals exist.
How do I evaluate a weakly supervised QA model fairly?
Use a held-out test set with human-verified answer spans that was not derived from the same weak-labeling pipeline as training. Standard metrics are Exact Match and F1 over the answer string.
Sources
- 1.Clark, C., & Gardner, M. (2018). Simple and Effective Multi-Paragraph Reading Comprehension. In Proceedings of ACL 2018, pp. 845–855. Association for Computational Linguistics.
- 2.Min, S., Chen, D., Hajishirzi, H., & Zettlemoyer, L. (2019). A Discrete Hard EM Approach for Weakly Supervised Question Answering. In Proceedings of EMNLP-IJCNLP 2019, pp. 2083–2093. Association for Computational Linguistics.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Weakly supervised question answering. ScholarGate. https://scholargate.app/deep-learning/weakly-supervised-question-answering