Bayesian Active Learning
Bayesian Active Learning (Query-by-Committee and BALD) · Also known as: BAL, Bayesian optimal experimental design for ML, BALD (Bayesian Active Learning by Disagreement), probabilistic active learning
Bayesian Active Learning (BAL) combines a probabilistic model with an active query strategy to identify the unlabeled examples that, once labeled, would most reduce model uncertainty. Instead of labeling data at random, BAL guides an oracle — typically a human annotator — toward the points where labeling will provide the greatest information gain, making it highly label-efficient.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Bayesian Active Learning when labeling is expensive or slow — medical image annotation, expert review, wet-lab experiments — and you have a large pool of unlabeled data but a tight labeling budget. It shines in scientific discovery settings (drug screening, materials science) where each oracle query has real cost. Avoid it when: labels are cheap and plentiful (passive random sampling suffices); the model class cannot represent calibrated uncertainty (plain neural networks without Bayesian treatment); the unlabeled pool is too small to gain from selective querying; or real-time annotation is impractical because the query loop cannot pause for an oracle.
Strengths & limitations
- Dramatically reduces the number of labeled examples needed to achieve a target accuracy.
- Provides calibrated uncertainty estimates alongside predictions, which are valuable in high-stakes domains.
- Principled information-theoretic acquisition functions (BALD, BatchBALD) avoid arbitrary heuristics.
- Naturally avoids querying redundant or already-certain regions of the input space.
- Applicable to diverse model classes: Gaussian processes, Bayesian neural networks, and approximate Bayesian methods (MC Dropout, deep ensembles).
- Requires the model to produce calibrated uncertainty; misspecified priors or overconfident approximations degrade query quality.
- Computationally expensive: posterior inference and scoring all unlabeled candidates at each round can be prohibitive for large pools and complex models.
- Batch selection without diversity correction (e.g., BatchBALD) leads to redundant queries.
- Performance advantage over random sampling can collapse when the initial labeled set is very small or the model class is poorly chosen.
- Assumes access to an oracle at each query round; interrupted or delayed annotation workflows break the loop.
Frequently asked
What is the difference between standard active learning and Bayesian active learning?
Standard active learning uses heuristic uncertainty measures such as least-confidence or margin sampling from a point-estimate model. Bayesian active learning grounds the query strategy in a full posterior distribution over model parameters, enabling principled information-theoretic acquisition functions and calibrated uncertainty that distinguish irreducible from learnable uncertainty.
Can I use Bayesian active learning with deep neural networks?
Yes, using approximate Bayesian inference. MC Dropout (Gal & Ghahramani, 2016) treats dropout at test time as approximate Bayesian inference and is widely used as a practical proxy. Deep ensembles are another well-calibrated option. Full Bayesian neural networks via variational inference or MCMC are more accurate but computationally demanding.
How many queries per round should I select?
Single-point (greedy) selection is theoretically optimal but slow. Batch selection (e.g., BatchBALD) picks a diverse batch that avoids redundancy, enabling parallel annotation. Batch sizes between 5 and 50 are common; the right size depends on annotation throughput and retraining cost.
Is Bayesian active learning always better than random labeling?
Not always. When the labeling budget is very large relative to the dataset, or when the model class is poorly calibrated, random sampling can match or beat BAL. The benefit is most pronounced in the low-label regime with a well-calibrated model and a large unlabeled pool.
What evaluation metrics should I use?
Plot learning curves — accuracy, F1, or AUC — against the number of labeled examples, comparing BAL to random baseline sampling. Also report expected calibration error (ECE) to confirm uncertainty estimates are meaningful. Evaluating at multiple budget checkpoints shows where BAL's advantage is largest.
Sources
- Houlsby, N., Huszár, F., Ghahramani, Z., & Lengyel, M. (2011). Bayesian Active Learning for Classification and Preference Learning. arXiv preprint arXiv:1112.5745. link ↗
- Settles, B. (2012). Active Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 6(1), 1–114. Morgan & Claypool. DOI: 10.2200/S00429ED1V01Y201207AIM018 ↗
How to cite this page
ScholarGate. (2026, June 3). Bayesian Active Learning (Query-by-Committee and BALD). ScholarGate. https://scholargate.app/en/machine-learning/bayesian-active-learning
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Active LearningMachine learning↔ compare
- Bayesian Logistic RegressionBayesian↔ compare
- Bayesian OptimizationOptimization↔ compare
- Few-shot LearningMachine learning↔ compare
- Gaussian ProcessMachine learning↔ compare
- Semi-supervised LearningMachine learning↔ compare