Bayesian Few-Shot Learning
Bayesian Few-Shot Learning (Meta-Learning with Bayesian Inference) · Also known as: Bayesian meta-learning, probabilistic few-shot learning, amortized Bayesian few-shot learning, Bayesian FSL
Bayesian few-shot learning combines Bayesian inference with meta-learning to enable a model to generalize from as few as one to five labeled examples per class. By treating task-specific parameters as random variables and learning an informative prior across many training tasks, the method produces calibrated uncertainty estimates alongside predictions — a key advantage over deterministic few-shot learners.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Bayesian few-shot learning when labeled data for a new class or task is extremely scarce — typically one to twenty examples — and calibrated uncertainty matters: medical diagnosis from rare conditions, drug discovery with limited assay data, personalized recommendation with cold-start users, or safety-critical classification where knowing the model is uncertain is as important as its prediction. The approach assumes that many structurally similar tasks (or classes) exist for meta-training; if no such related-task data is available, the method loses its core advantage. Do not use it as a drop-in replacement for standard supervised learning when ample labeled data exists — a well-tuned Random Forest or gradient boosting model will typically outperform it in that regime.
Strengths & limitations
- Principled uncertainty quantification: predictions come with calibrated confidence intervals, not just point estimates.
- Rapid adaptation to new classes from as few as one to five labeled examples per class.
- Exploits structural similarity across tasks through an informative learned prior.
- Amortized inference networks make Bayesian updating fast at test time despite formal complexity.
- Strong performance in safety-critical domains where overconfident wrong predictions are especially harmful.
- Requires a large and diverse meta-training set of related tasks; performance degrades sharply if meta-training tasks do not resemble the target task.
- Architectural complexity is higher than standard few-shot learners; amortized inference networks add design and tuning burden.
- Approximation methods (variational inference, MCMC) introduce their own biases and may not converge to the true posterior.
- Computational cost at meta-training time is substantially higher than deterministic counterparts such as Prototypical Networks.
Frequently asked
How is Bayesian few-shot learning different from standard few-shot learning?
Standard few-shot methods like Prototypical Networks produce a single point prediction. Bayesian few-shot learning additionally models uncertainty over task parameters, yielding a predictive distribution — so you get both a label and a calibrated confidence estimate.
What is amortized inference in this context?
Rather than running iterative MCMC or variational optimization for each new task, an inference network is trained to map the support set directly to posterior parameters in one forward pass. This makes Bayesian updating fast enough to use at test time.
How many meta-training tasks are typically needed?
Practical systems use thousands to tens of thousands of episodes drawn from a large pool of base classes. The exact number depends on task diversity and model capacity, but the meta-training set should be substantially larger than the target task pool.
Can I apply this without a GPU?
Meta-training is compute-intensive and practically requires a GPU. Inference (applying the trained meta-learner to a new task) can be done on CPU but is slow for large models. For resource-constrained settings, consider simpler prototypical or matching-network baselines.
How do I evaluate calibration?
Compute the Expected Calibration Error (ECE) alongside accuracy: bin predictions by confidence, then measure the gap between average confidence and average accuracy within each bin. A well-calibrated Bayesian few-shot model should show small ECE.
Sources
- Gordon, J., Bronskill, J., Bauer, M., Nowozin, S. & Turner, R. E. (2019). Meta-Learning Probabilistic Inference for Prediction. International Conference on Learning Representations (ICLR 2019). link ↗
- Finn, C., Xu, K. & Levine, S. (2018). Probabilistic Model-Agnostic Meta-Learning. Advances in Neural Information Processing Systems (NeurIPS 2018), 31. link ↗
How to cite this page
ScholarGate. (2026, June 3). Bayesian Few-Shot Learning (Meta-Learning with Bayesian Inference). ScholarGate. https://scholargate.app/en/machine-learning/bayesian-few-shot-learning
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Transfer LearningMachine learning↔ compare
- Few-shot LearningMachine learning↔ compare
- Gaussian ProcessMachine learning↔ compare
- Semi-supervised Few-shot LearningMachine learning↔ compare
- Transfer LearningMachine learning↔ compare