Optimizing the Collaboration Structure in Cross-Silo Federated Learning

Wenxuan Bao, Haohan Wang, Jun Wu, Jingrui He
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:1718-1736, 2023.

Abstract

In federated learning (FL), multiple clients collaborate to train machine learning models together while keeping their data decentralized. Through utilizing more training data, FL suffers from the potential negative transfer problem: the global FL model may even perform worse than the models trained with local data only. In this paper, we propose FedCollab, a novel FL framework that alleviates negative transfer by clustering clients into non-overlapping coalitions based on their distribution distances and data quantities. As a result, each client only collaborates with the clients having similar data distributions, and tends to collaborate with more clients when it has less data. We evaluate our framework with a variety of datasets, models, and types of non-IIDness. Our results demonstrate that FedCollab effectively mitigates negative transfer across a wide range of FL algorithms and consistently outperforms other clustered FL algorithms.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-bao23b, title = {Optimizing the Collaboration Structure in Cross-Silo Federated Learning}, author = {Bao, Wenxuan and Wang, Haohan and Wu, Jun and He, Jingrui}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {1718--1736}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/bao23b/bao23b.pdf}, url = {https://proceedings.mlr.press/v202/bao23b.html}, abstract = {In federated learning (FL), multiple clients collaborate to train machine learning models together while keeping their data decentralized. Through utilizing more training data, FL suffers from the potential negative transfer problem: the global FL model may even perform worse than the models trained with local data only. In this paper, we propose FedCollab, a novel FL framework that alleviates negative transfer by clustering clients into non-overlapping coalitions based on their distribution distances and data quantities. As a result, each client only collaborates with the clients having similar data distributions, and tends to collaborate with more clients when it has less data. We evaluate our framework with a variety of datasets, models, and types of non-IIDness. Our results demonstrate that FedCollab effectively mitigates negative transfer across a wide range of FL algorithms and consistently outperforms other clustered FL algorithms.} }
Endnote
%0 Conference Paper %T Optimizing the Collaboration Structure in Cross-Silo Federated Learning %A Wenxuan Bao %A Haohan Wang %A Jun Wu %A Jingrui He %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-bao23b %I PMLR %P 1718--1736 %U https://proceedings.mlr.press/v202/bao23b.html %V 202 %X In federated learning (FL), multiple clients collaborate to train machine learning models together while keeping their data decentralized. Through utilizing more training data, FL suffers from the potential negative transfer problem: the global FL model may even perform worse than the models trained with local data only. In this paper, we propose FedCollab, a novel FL framework that alleviates negative transfer by clustering clients into non-overlapping coalitions based on their distribution distances and data quantities. As a result, each client only collaborates with the clients having similar data distributions, and tends to collaborate with more clients when it has less data. We evaluate our framework with a variety of datasets, models, and types of non-IIDness. Our results demonstrate that FedCollab effectively mitigates negative transfer across a wide range of FL algorithms and consistently outperforms other clustered FL algorithms.
APA
Bao, W., Wang, H., Wu, J. & He, J.. (2023). Optimizing the Collaboration Structure in Cross-Silo Federated Learning. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:1718-1736 Available from https://proceedings.mlr.press/v202/bao23b.html.

Related Material