Ensemble Active Learning
Ensemble-Based Active Learning (Query by Committee and Variants) · Also known as: Query by Committee, QBC active learning, committee-based active learning, ensemble query strategy
Ensemble Active Learning combines a committee of diverse models with an active learning loop to select the most informative unlabeled examples for labeling. Rooted in the Query by Committee framework introduced by Seung et al. (1992), it uses disagreement among committee members as a signal for uncertainty, reducing the number of labeled examples needed to achieve strong predictive performance.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Ensemble Active Learning is well suited when labeling is expensive or slow — such as medical annotation, expert review, or physical experimentation — and a pool of unlabeled examples is available. It excels when the decision boundary is complex and uncertain, and the committee's diversity can meaningfully reflect that uncertainty. Use it with at least a small labeled seed (typically 10–50 examples per class) and a committee of 3–10 diverse members. Avoid it when labeling is cheap (passive learning suffices), when the pool is very small, when ground truth is unreliable (oracle noise undermines the strategy), or when real-time prediction is required without the capacity to pause for labeling rounds.
Strengths & limitations
- Drastically reduces the number of labeled examples needed to reach a given accuracy, lowering annotation cost.
- Ensemble disagreement provides a principled, model-agnostic uncertainty estimate without requiring probability calibration.
- Applicable to any base learner that produces predictions or class probabilities, including decision trees, SVMs, and neural networks.
- Naturally identifies hard or boundary-region examples, which are often the most scientifically interesting cases.
- Can be combined with diversity-promoting sampling to avoid redundant queries in dense unlabeled clusters.
- Requires managing an iterative labeling loop, which adds implementation overhead compared to passive learning.
- Committee diversity must be carefully designed; insufficiently diverse committees collapse to single-model uncertainty and lose the ensemble benefit.
- Query strategies can be biased toward outliers or noisy examples near spurious decision boundaries, wasting labeling budget.
- In small labeled-set regimes, committee members may all be poorly calibrated, making disagreement scores unreliable.
- The computational cost scales with committee size and pool size, which can be prohibitive for very large unlabeled pools.
Frequently asked
How many committee members should I use?
Research suggests that committees of 3 to 10 diverse members offer a good balance between disagreement reliability and computational cost. Very small committees (2 members) produce noisy disagreement estimates; very large committees are computationally expensive without commensurate gains.
What disagreement metric should I choose?
Vote entropy is the most common and interpretable: it measures the spread of class votes across committee members. For probabilistic models, KL divergence between member posteriors is more principled. For regression, variance of predictions is the natural choice.
Can ensemble active learning handle class imbalance?
It can partially help by directing queries toward uncertain boundary regions where minority class examples may cluster, but it should be combined with class-weighting or resampling strategies to avoid over-querying the majority class boundary.
Is Query by Committee the only ensemble active learning strategy?
No. Other ensemble-based strategies include Bootstrap Aggregation Active Learning, Random Committee, and ensemble methods for deep learning (e.g., MC Dropout ensembles). QBC is the classical and most widely cited framework.
When does ensemble active learning fail to outperform random sampling?
When the unlabeled pool is very small (so all strategies converge quickly), when the oracle is noisy (queried labels are unreliable), or when the committee lacks diversity (all members agree everywhere). In those cases, random or stratified sampling can be competitive.
Sources
- Seung, H. S., Opper, M., & Sompolinsky, H. (1992). Query by committee. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory (COLT 1992), pp. 287–294. ACM. link ↗
- Settles, B. (2009). Active Learning Literature Survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison. link ↗
How to cite this page
ScholarGate. (2026, June 3). Ensemble-Based Active Learning (Query by Committee and Variants). ScholarGate. https://scholargate.app/en/machine-learning/ensemble-active-learning
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Active LearningMachine learning↔ compare
- BoostingMachine learning↔ compare
- Random ForestMachine learning↔ compare
- Semi-supervised LearningMachine learning↔ compare
- Voting EnsembleMachine learning↔ compare