Active Learning
Active Learning (Human-in-the-Loop) · Also known as: Query Learning, Optimal Experimental Design (ML context), Pool-Based Active Learning, Aktif Öğrenme
Active learning is an iterative machine-learning paradigm in which a learning algorithm selectively queries an oracle — typically a human annotator — for labels on the most informative unlabeled examples. Formalized by Burr Settles in his seminal 2009 literature survey, active learning addresses the practical bottleneck of annotation cost by achieving high model accuracy with far fewer labeled examples than passive supervised learning requires.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+18 more
When to use it
Active learning is most beneficial when unlabeled data is abundant but expert annotation is expensive, slow, or scarce — common in medical imaging, legal document classification, scientific data curation, and natural language processing. It assumes oracle availability and that the query strategy can meaningfully identify informative examples. It is less suitable when annotation cost is negligible, the unlabeled pool is small, or the task distribution shifts substantially between queries. Alternatives include semi-supervised learning (when labels cannot be obtained interactively) and self-training (when oracle queries are infeasible).
Strengths & limitations
- Drastically reduces labeling cost by focusing annotation effort on maximally informative examples.
- Applicable to any supervised learner — classification, regression, structured prediction — without modifying the underlying model architecture.
- Theoretically grounded: uncertainty sampling and query-by-committee have PAC-learning and Bayesian justifications.
- Compatible with crowdsourcing workflows and modern annotation platforms.
- Query strategies can introduce sampling bias, causing the labeled set to be non-representative of the true data distribution.
- Performance gains depend heavily on the choice of query strategy, which is task- and model-specific.
- Retraining the model after each query is computationally expensive for large neural networks.
- Oracle noise (annotator disagreement or errors) can degrade performance if not explicitly modeled.
Frequently asked
How is active learning different from semi-supervised learning?
Semi-supervised learning exploits the structure of unlabeled data (e.g., clustering, manifold assumptions) without querying an oracle, updating the model using pseudo-labels or consistency regularization. Active learning explicitly requests true labels from an oracle for selected examples. The two approaches are complementary and can be combined.
Which query strategy should I use in practice?
Uncertainty sampling is the most widely used baseline and works well for binary and multi-class classifiers. Query-by-committee is preferable when an ensemble is already available. Expected model change or BALD (Bayesian Active Learning by Disagreement) tend to perform better for deep models but are computationally heavier. Start with uncertainty sampling and benchmark against alternatives.
Can active learning be applied to regression tasks?
Yes. For regression, informativeness is typically measured by predictive variance (e.g., from a Gaussian process or ensemble) rather than class probability entropy. The learner queries points where variance is highest. This is standard practice in Bayesian optimization and surrogate-model-based design of experiments.
Sources
- Settles, B. (2009). Active learning literature survey. University of Wisconsin-Madison Computer Sciences Technical Report 1648. link ↗
How to cite this page
ScholarGate. (2026, June 2). Active Learning (Human-in-the-Loop). ScholarGate. https://scholargate.app/en/machine-learning/active-learning
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Conformal PredictionMachine learning↔ compare
- Uncertainty QuantificationSimulation↔ compare