Ensemble Federated Learning
Ensemble Federated Learning (Federated Ensemble Aggregation) · Also known as: federated ensemble learning, EFL, federated model ensembling, federated multi-model aggregation
Ensemble Federated Learning combines the privacy-preserving distribution of federated learning with ensemble aggregation: each participating client trains its own local model on private data, and the server aggregates predictions — or model parameters — from all clients using ensemble strategies such as voting, averaging, or stacking, instead of simple parameter averaging alone.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Ensemble Federated Learning when data cannot be centralised due to privacy, regulatory, or ownership constraints, and when local data distributions differ substantially across clients (non-IID setting), making simple parameter averaging (FedAvg) insufficient. It is particularly suited to healthcare, finance, and IoT domains with heterogeneous clients and moderate numbers of participating nodes (5–100). Avoid it when all data can be pooled freely (a standard ensemble is simpler and faster), when the number of clients is so large that collecting model outputs is impractical, or when communication bandwidth is extremely limited and transmitting full model outputs is prohibitive.
Strengths & limitations
- Preserves raw-data privacy by keeping all training data on local clients at all times.
- Handles non-IID data better than FedAvg because local models are not averaged into a single global parameter set.
- Ensemble aggregation improves generalisation and robustness compared to single-model federated learning.
- Compatible with heterogeneous client architectures — different clients can use different model types.
- Naturally resistant to single-client failure or dropout because the ensemble can proceed with fewer models.
- Communication cost grows with the number of clients, especially when transmitting full model parameters instead of predictions.
- Designing an effective ensemble aggregation rule (weights, meta-learner) requires a shared validation set at the server, which may not always be available.
- Clients with low-quality or very small local datasets can degrade ensemble quality if not down-weighted.
- More complex to implement and audit than standard federated averaging.
Frequently asked
How is Ensemble Federated Learning different from FedAvg?
FedAvg averages model weights across clients into a single global model. Ensemble Federated Learning keeps each client's model separate and aggregates their predictions or outputs at inference time, preserving local specialisation and improving non-IID performance.
Does transmitting model outputs violate privacy?
Transmitting predictions rather than raw data substantially reduces leakage, but it is not perfectly private. For strong privacy guarantees, combine Ensemble Federated Learning with differential privacy noise injection or secure aggregation protocols.
What aggregation rule should I use?
For classification, soft-voting with client weights proportional to local validation accuracy is a common default. For regression, weighted averaging works well. If a shared server-side validation set is available, a lightweight stacking meta-learner often outperforms fixed-weight rules.
How many clients are needed?
Ensemble benefits typically become meaningful at five or more clients. Very large-scale federations (thousands of clients) usually favour communication-efficient approaches such as federated distillation rather than full ensemble aggregation.
Can clients use different model types?
Yes. Because aggregation happens at the prediction level rather than the parameter level, different clients can use neural networks, gradient boosters, or linear models simultaneously, making the method well-suited to heterogeneous environments.
Sources
- McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 54, 1273–1282. link ↗
- Chen, Y., Qin, X., Wang, J., Yu, C., & Gao, W. (2021). FedHealth: A federated transfer learning framework for wearable healthcare. IEEE Intelligent Systems, 35(4), 83–93. DOI: 10.1109/MIS.2020.2988604 ↗
How to cite this page
ScholarGate. (2026, June 3). Ensemble Federated Learning (Federated Ensemble Aggregation). ScholarGate. https://scholargate.app/en/machine-learning/ensemble-federated-learning
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- BaggingMachine learning↔ compare
- BoostingMachine learning↔ compare
- Federated LearningPrivacy↔ compare
- StackingMachine learning↔ compare
- Transfer LearningMachine learning↔ compare
- Voting EnsembleMachine learning↔ compare