EFL: Elastic Federated Learning on Non-IID Data

Zichen Ma, Yu Lu, Wenye Li, Shuguang Cui
Proceedings of The 1st Conference on Lifelong Learning Agents, PMLR 199:92-115, 2022.

Abstract

Federated learning involves training machine learning models over devices or data silos, such as edge processors or data warehouses while keeping the data local. However, training in heterogeneous and potentially massive networks introduces bias into the system, originating from the non-IID data and the low participation rate. In this paper, we propose Elastic Federated Learning (EFL), an unbiased federated training framework capable of tackling the heterogeneity in the system. EFL extends lifelong learning to realistic federated settings, makes the most informative parameters less volatile during training, and utilizes incomplete local updates. It is also an efficient and effective algorithm that compresses upstream and downstream communications with a convergence guarantee. We empirically demonstrate the efficacy of our framework on a variety of non-IID datasets and show the competitive performance of the algorithm on robustness and efficiency.

Cite this Paper

BibTeX
@InProceedings{pmlr-v199-ma22a, title = {EFL: Elastic Federated Learning on Non-IID Data}, author = {Ma, Zichen and Lu, Yu and Li, Wenye and Cui, Shuguang}, booktitle = {Proceedings of The 1st Conference on Lifelong Learning Agents}, pages = {92--115}, year = {2022}, editor = {Chandar, Sarath and Pascanu, Razvan and Precup, Doina}, volume = {199}, series = {Proceedings of Machine Learning Research}, month = {22--24 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v199/ma22a/ma22a.pdf}, url = {https://proceedings.mlr.press/v199/ma22a.html}, abstract = {Federated learning involves training machine learning models over devices or data silos, such as edge processors or data warehouses while keeping the data local. However, training in heterogeneous and potentially massive networks introduces bias into the system, originating from the non-IID data and the low participation rate. In this paper, we propose Elastic Federated Learning (EFL), an unbiased federated training framework capable of tackling the heterogeneity in the system. EFL extends lifelong learning to realistic federated settings, makes the most informative parameters less volatile during training, and utilizes incomplete local updates. It is also an efficient and effective algorithm that compresses upstream and downstream communications with a convergence guarantee. We empirically demonstrate the efficacy of our framework on a variety of non-IID datasets and show the competitive performance of the algorithm on robustness and efficiency.} }
Endnote
%0 Conference Paper %T EFL: Elastic Federated Learning on Non-IID Data %A Zichen Ma %A Yu Lu %A Wenye Li %A Shuguang Cui %B Proceedings of The 1st Conference on Lifelong Learning Agents %C Proceedings of Machine Learning Research %D 2022 %E Sarath Chandar %E Razvan Pascanu %E Doina Precup %F pmlr-v199-ma22a %I PMLR %P 92--115 %U https://proceedings.mlr.press/v199/ma22a.html %V 199 %X Federated learning involves training machine learning models over devices or data silos, such as edge processors or data warehouses while keeping the data local. However, training in heterogeneous and potentially massive networks introduces bias into the system, originating from the non-IID data and the low participation rate. In this paper, we propose Elastic Federated Learning (EFL), an unbiased federated training framework capable of tackling the heterogeneity in the system. EFL extends lifelong learning to realistic federated settings, makes the most informative parameters less volatile during training, and utilizes incomplete local updates. It is also an efficient and effective algorithm that compresses upstream and downstream communications with a convergence guarantee. We empirically demonstrate the efficacy of our framework on a variety of non-IID datasets and show the competitive performance of the algorithm on robustness and efficiency.
APA
Ma, Z., Lu, Y., Li, W. & Cui, S.. (2022). EFL: Elastic Federated Learning on Non-IID Data. Proceedings of The 1st Conference on Lifelong Learning Agents, in Proceedings of Machine Learning Research 199:92-115 Available from https://proceedings.mlr.press/v199/ma22a.html.

Related Material