Adaptive Group Personalization for Federated Mutual Transfer Learning

Haoqing Xu, Dian Shen, Meng Wang, Beilun Wang
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:55225-55240, 2024.

Abstract

Mutual transfer learning aims to improve prediction with knowledge from related domains. Recently, federated learning is applied in this field to address the communication and privacy concerns. However, previous clustered federated learning (CFL) solutions lack theoretical guarantee of learnability recovery and require time-consuming hyper-parameter tuning, while centralized mutual transfer learning methods lack adaptability to concept drifts. In this paper, we propose the Adaptive Group Personalization method (AdaGrP) to overcome these challenges. We adaptively decide the recovery threshold with a nonparametric method, adaptive threshold correction, for tuning-free solution with relaxed condition. Theoretical results guarantee the perfect learnability recovery with the corrected threshold. Empirical results show AdaGrP achieves 16.9% average improvement in learnability structure recovery compared with state-of-the-art CFL baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-xu24u, title = {Adaptive Group Personalization for Federated Mutual Transfer Learning}, author = {Xu, Haoqing and Shen, Dian and Wang, Meng and Wang, Beilun}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {55225--55240}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/xu24u/xu24u.pdf}, url = {https://proceedings.mlr.press/v235/xu24u.html}, abstract = {Mutual transfer learning aims to improve prediction with knowledge from related domains. Recently, federated learning is applied in this field to address the communication and privacy concerns. However, previous clustered federated learning (CFL) solutions lack theoretical guarantee of learnability recovery and require time-consuming hyper-parameter tuning, while centralized mutual transfer learning methods lack adaptability to concept drifts. In this paper, we propose the Adaptive Group Personalization method (AdaGrP) to overcome these challenges. We adaptively decide the recovery threshold with a nonparametric method, adaptive threshold correction, for tuning-free solution with relaxed condition. Theoretical results guarantee the perfect learnability recovery with the corrected threshold. Empirical results show AdaGrP achieves 16.9% average improvement in learnability structure recovery compared with state-of-the-art CFL baselines.} }
Endnote
%0 Conference Paper %T Adaptive Group Personalization for Federated Mutual Transfer Learning %A Haoqing Xu %A Dian Shen %A Meng Wang %A Beilun Wang %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-xu24u %I PMLR %P 55225--55240 %U https://proceedings.mlr.press/v235/xu24u.html %V 235 %X Mutual transfer learning aims to improve prediction with knowledge from related domains. Recently, federated learning is applied in this field to address the communication and privacy concerns. However, previous clustered federated learning (CFL) solutions lack theoretical guarantee of learnability recovery and require time-consuming hyper-parameter tuning, while centralized mutual transfer learning methods lack adaptability to concept drifts. In this paper, we propose the Adaptive Group Personalization method (AdaGrP) to overcome these challenges. We adaptively decide the recovery threshold with a nonparametric method, adaptive threshold correction, for tuning-free solution with relaxed condition. Theoretical results guarantee the perfect learnability recovery with the corrected threshold. Empirical results show AdaGrP achieves 16.9% average improvement in learnability structure recovery compared with state-of-the-art CFL baselines.
APA
Xu, H., Shen, D., Wang, M. & Wang, B.. (2024). Adaptive Group Personalization for Federated Mutual Transfer Learning. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:55225-55240 Available from https://proceedings.mlr.press/v235/xu24u.html.

Related Material