Active Learning Voting Ensemble
Active Learning with Voting Ensemble (Query by Committee) · Also known as: Query by Committee, QBC, active ensemble learning, committee-based active learning
Active Learning Voting Ensemble — formally known as Query by Committee — is an active learning strategy that trains a committee of diverse models and selects the unlabeled examples where the committee members disagree most for human annotation. By focusing labeling effort on the most informative points, it achieves high accuracy with far fewer labeled examples than passive learning requires.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use active learning voting ensemble when labeling is expensive or time-consuming — clinical annotation, legal document tagging, specialized image review — and a large pool of unlabeled data is available alongside a small labeled seed set. It works well for classification tasks where committee disagreement is a meaningful signal, and when at least 3–5 diverse models can be fielded. Avoid it when labels are cheap and plentiful (passive learning on the full dataset will then be equally or more efficient), when the unlabeled pool is very small, when real-time online annotation is infeasible, or when a single interpretable model is required rather than an ensemble.
Strengths & limitations
- Dramatically reduces the number of labeled examples needed to reach a target accuracy compared with random sampling.
- Ensemble voting provides the dual benefit of lower variance predictions at inference time.
- Model-agnostic: any set of diverse classifiers can form the committee.
- The disagreement score provides a built-in uncertainty signal that is easy to monitor and visualize.
- Well-founded theoretically through the Query by Committee framework introduced in 1992.
- Requires an annotation loop, which adds infrastructure complexity: a pool manager, oracle interface, and retraining pipeline.
- Performance depends heavily on committee diversity; a homogeneous committee produces uninformative disagreement.
- Pool-based active learning assumes the full unlabeled pool fits in memory; very large pools require approximations.
- Convergence is not guaranteed if the oracle makes systematic labeling errors.
Frequently asked
How many committee members should I use?
Three to ten diverse models is a common practical range. Fewer members yield noisier disagreement estimates; more members increase compute costs. Empirically, five diverse classifiers often provide a good balance between stability and efficiency.
How do I measure disagreement in the committee?
Vote entropy is the most common measure: H = -sum_c p(c) log p(c) where p(c) is the fraction of committee members predicting class c. For probabilistic models, average KL divergence from the mean prediction is an alternative. Higher scores indicate more informative samples.
Can I use this with deep learning models?
Yes. Deep ensembles or MC-dropout approximations can serve as the committee. The main challenge is retraining cost — strategies like fine-tuning a shared backbone with diverse heads reduce this considerably.
What if my annotation budget is very tight — say, fewer than 20 labels?
With very few labels, random sampling or stratified seed selection often performs comparably to QBC because all methods start from nearly the same information-poor baseline. QBC's advantage emerges more clearly once the seed set has at least 20–50 examples per class.
Does active learning voting ensemble work for regression?
Yes, with disagreement measured as the variance or standard deviation of committee predictions rather than vote entropy. The same query-retrain loop applies; the final output is the mean of committee predictions.
Sources
- Seung, H. S., Opper, M., & Sompolinsky, H. (1992). Query by committee. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory (COLT '92), pp. 287–294. ACM. DOI: 10.1145/130385.130417 ↗
- Settles, B. (2009). Active Learning Literature Survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison. link ↗
How to cite this page
ScholarGate. (2026, June 3). Active Learning with Voting Ensemble (Query by Committee). ScholarGate. https://scholargate.app/en/machine-learning/active-learning-voting-ensemble
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Active LearningMachine learning↔ compare
- BaggingMachine learning↔ compare
- BoostingMachine learning↔ compare
- Semi-supervised LearningMachine learning↔ compare
- Voting EnsembleMachine learning↔ compare