Federated Learning under Distributed Concept Drift

Ellango Jothimurugesan, Kevin Hsieh, Jianyu Wang, Gauri Joshi, Phillip B. Gibbons
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:5834-5853, 2023.

Abstract

Federated Learning (FL) under distributed concept drift is a largely unexplored area. Although concept drift is itself a well-studied phenomenon, it poses particular challenges for FL, because drifts arise staggered in time and space (across clients). Our work is the first to explicitly study data heterogeneity in both dimensions. We first demonstrate that prior solutions to drift adaptation, with their single global model, are ill-suited to staggered drifts, necessitating multiple-model solutions. We identify the problem of drift adaptation as a time-varying clustering problem, and we propose two new clustering algorithms for reacting to drifts based on local drift detection and hierarchical clustering. Empirical evaluation shows that our solutions achieve significantly higher accuracy than existing baselines, and are comparable to an idealized algorithm with oracle knowledge of the ground-truth clustering of clients to concepts at each time step.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-jothimurugesan23a, title = {Federated Learning under Distributed Concept Drift}, author = {Jothimurugesan, Ellango and Hsieh, Kevin and Wang, Jianyu and Joshi, Gauri and Gibbons, Phillip B.}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {5834--5853}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/jothimurugesan23a/jothimurugesan23a.pdf}, url = {https://proceedings.mlr.press/v206/jothimurugesan23a.html}, abstract = {Federated Learning (FL) under distributed concept drift is a largely unexplored area. Although concept drift is itself a well-studied phenomenon, it poses particular challenges for FL, because drifts arise staggered in time and space (across clients). Our work is the first to explicitly study data heterogeneity in both dimensions. We first demonstrate that prior solutions to drift adaptation, with their single global model, are ill-suited to staggered drifts, necessitating multiple-model solutions. We identify the problem of drift adaptation as a time-varying clustering problem, and we propose two new clustering algorithms for reacting to drifts based on local drift detection and hierarchical clustering. Empirical evaluation shows that our solutions achieve significantly higher accuracy than existing baselines, and are comparable to an idealized algorithm with oracle knowledge of the ground-truth clustering of clients to concepts at each time step.} }
Endnote
%0 Conference Paper %T Federated Learning under Distributed Concept Drift %A Ellango Jothimurugesan %A Kevin Hsieh %A Jianyu Wang %A Gauri Joshi %A Phillip B. Gibbons %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-jothimurugesan23a %I PMLR %P 5834--5853 %U https://proceedings.mlr.press/v206/jothimurugesan23a.html %V 206 %X Federated Learning (FL) under distributed concept drift is a largely unexplored area. Although concept drift is itself a well-studied phenomenon, it poses particular challenges for FL, because drifts arise staggered in time and space (across clients). Our work is the first to explicitly study data heterogeneity in both dimensions. We first demonstrate that prior solutions to drift adaptation, with their single global model, are ill-suited to staggered drifts, necessitating multiple-model solutions. We identify the problem of drift adaptation as a time-varying clustering problem, and we propose two new clustering algorithms for reacting to drifts based on local drift detection and hierarchical clustering. Empirical evaluation shows that our solutions achieve significantly higher accuracy than existing baselines, and are comparable to an idealized algorithm with oracle knowledge of the ground-truth clustering of clients to concepts at each time step.
APA
Jothimurugesan, E., Hsieh, K., Wang, J., Joshi, G. & Gibbons, P.B.. (2023). Federated Learning under Distributed Concept Drift. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:5834-5853 Available from https://proceedings.mlr.press/v206/jothimurugesan23a.html.

Related Material