Fast Heterogeneous Federated Learning with Hybrid Client Selection

Duanxiao Song, Guangyuan Shen, Dehong Gao, Libin Yang, Xukai Zhou, Shirui Pan, Wei Lou, Fang Zhou
Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, PMLR 216:2006-2015, 2023.

Abstract

Client selection schemes are widely adopted to handle the communication-efficient problems in recent studies of Federated Learning (FL). However, the large variance of the model updates aggregated from the randomly-selected unrepresentative subsets directly slows the FL convergence. We present a novel clustering-based client selection scheme to accelerate the FL convergence by variance reduction. Simple yet effective schemes are designed to improve the clustering effect and control the effect fluctuation, therefore, generating the client subset with certain representativeness of sampling. Theoretically, we demonstrate the improvement of the proposed scheme in variance reduction. We also present the tighter convergence guarantee of the proposed method thanks to the variance reduction. Experimental results confirm the exceed efficiency of our scheme compared to alternatives.

Cite this Paper

BibTeX
@InProceedings{pmlr-v216-song23b, title = {Fast Heterogeneous Federated Learning with Hybrid Client Selection}, author = {Song, Duanxiao and Shen, Guangyuan and Gao, Dehong and Yang, Libin and Zhou, Xukai and Pan, Shirui and Lou, Wei and Zhou, Fang}, booktitle = {Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence}, pages = {2006--2015}, year = {2023}, editor = {Evans, Robin J. and Shpitser, Ilya}, volume = {216}, series = {Proceedings of Machine Learning Research}, month = {31 Jul--04 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v216/song23b/song23b.pdf}, url = {https://proceedings.mlr.press/v216/song23b.html}, abstract = {Client selection schemes are widely adopted to handle the communication-efficient problems in recent studies of Federated Learning (FL). However, the large variance of the model updates aggregated from the randomly-selected unrepresentative subsets directly slows the FL convergence. We present a novel clustering-based client selection scheme to accelerate the FL convergence by variance reduction. Simple yet effective schemes are designed to improve the clustering effect and control the effect fluctuation, therefore, generating the client subset with certain representativeness of sampling. Theoretically, we demonstrate the improvement of the proposed scheme in variance reduction. We also present the tighter convergence guarantee of the proposed method thanks to the variance reduction. Experimental results confirm the exceed efficiency of our scheme compared to alternatives.} }
Endnote
%0 Conference Paper %T Fast Heterogeneous Federated Learning with Hybrid Client Selection %A Duanxiao Song %A Guangyuan Shen %A Dehong Gao %A Libin Yang %A Xukai Zhou %A Shirui Pan %A Wei Lou %A Fang Zhou %B Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2023 %E Robin J. Evans %E Ilya Shpitser %F pmlr-v216-song23b %I PMLR %P 2006--2015 %U https://proceedings.mlr.press/v216/song23b.html %V 216 %X Client selection schemes are widely adopted to handle the communication-efficient problems in recent studies of Federated Learning (FL). However, the large variance of the model updates aggregated from the randomly-selected unrepresentative subsets directly slows the FL convergence. We present a novel clustering-based client selection scheme to accelerate the FL convergence by variance reduction. Simple yet effective schemes are designed to improve the clustering effect and control the effect fluctuation, therefore, generating the client subset with certain representativeness of sampling. Theoretically, we demonstrate the improvement of the proposed scheme in variance reduction. We also present the tighter convergence guarantee of the proposed method thanks to the variance reduction. Experimental results confirm the exceed efficiency of our scheme compared to alternatives.
APA
Song, D., Shen, G., Gao, D., Yang, L., Zhou, X., Pan, S., Lou, W. & Zhou, F.. (2023). Fast Heterogeneous Federated Learning with Hybrid Client Selection. Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 216:2006-2015 Available from https://proceedings.mlr.press/v216/song23b.html.

Related Material