Ensemble Few-Shot Learning
Ensemble Methods for Few-Shot Learning · Also known as: ensemble few-shot classification, multi-model few-shot learning, few-shot ensemble, cooperative few-shot ensemble
Ensemble Few-Shot Learning combines multiple few-shot models — such as prototypical networks or embedding learners — to classify new classes from only one to a handful of labeled examples. By enforcing diversity among base learners and aggregating their predictions, the ensemble consistently outperforms any single few-shot model in accuracy and robustness, especially under severe label scarcity.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use ensemble few-shot learning when labeled data is extremely scarce (1–20 examples per class), when a single few-shot model's accuracy is too variable across runs, or when deployment requires higher confidence and robustness than a single model can provide. It is especially suited to image recognition of rare categories, medical imaging with limited annotations, and natural language tasks with novel intents. Do not use it when computational budget is tight (M models cost M times the inference compute), when the few-shot task distribution shifts dramatically between meta-training and test (domain gap is better addressed by domain-adaptive meta-learning), or when interpretability of individual predictions is required.
Strengths & limitations
- Consistently reduces prediction variance compared to any single few-shot model without requiring more labeled data.
- Works on top of any metric-based or optimization-based few-shot backbone, making it a modular improvement.
- Soft aggregation of probabilities yields well-calibrated confidence scores useful for downstream thresholding.
- Diversity enforcement during training acts as an implicit regularizer, reducing overfitting to support-set noise.
- Robust to unlucky random initializations that occasionally cause a single model to collapse.
- Inference cost scales linearly with the number of base learners, which can be prohibitive on edge devices.
- Gains diminish rapidly after 5–10 diverse base learners; maintaining true diversity is non-trivial.
- Requires careful episodic meta-training infrastructure that is more complex than standard supervised training.
- Does not address domain shift between meta-training and deployment; ensemble variance reduction does not substitute for domain adaptation.
Frequently asked
How many base learners should the ensemble have?
Empirically, gains saturate around 5–10 diverse learners. Beyond that, the marginal accuracy improvement rarely justifies the added inference cost. Start with 5 and increase only if variance across test episodes remains high.
Does ensemble few-shot learning require more labeled data than a single model?
No. The ensemble uses the same meta-training set as a single model. The diversity comes from different initializations, augmentation policies, or architecture variants, not from additional labels.
How is diversity measured and enforced?
Diversity is typically measured as disagreement rate between models on query examples. It can be enforced implicitly via different random seeds or explicitly via a cooperation loss that penalizes models for agreeing on wrong predictions, as proposed by Dvornik et al. (2019).
Is soft averaging always better than hard majority voting?
In practice, yes — soft averaging preserves calibration and handles near-tie situations more gracefully. Hard voting should only be considered when base learners produce binary outputs and probability estimates are unavailable.
Can ensemble few-shot learning be combined with data augmentation or self-supervised pre-training?
Yes, and this is a common practice. Self-supervised pre-training on unlabeled data (e.g., SimCLR, DINO) produces strong feature representations, and running an ensemble on top of such representations consistently outperforms ensembles built on purely supervised backbones.
Sources
- Dvornik, N., Schmid, C., & Mairal, J. (2019). Diversity with Cooperation: Ensemble Methods for Few-Shot Classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3716–3725. link ↗
- Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a Few Examples: A Survey on Few-Shot Learning. ACM Computing Surveys, 53(3), 1–34. DOI: 10.1145/3386252 ↗
How to cite this page
ScholarGate. (2026, June 3). Ensemble Methods for Few-Shot Learning. ScholarGate. https://scholargate.app/en/machine-learning/ensemble-few-shot-learning
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- BoostingMachine learning↔ compare
- Few-shot LearningMachine learning↔ compare
- Semi-supervised Few-shot LearningMachine learning↔ compare
- Transfer LearningMachine learning↔ compare
- Voting EnsembleMachine learning↔ compare