FedDuA: Doubly Adaptive Federated Learning

Shokichi Takakura, Seng Pei Liew, Satoshi Hasegawa
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:586-594, 2026.

Abstract

Federated learning is a distributed learning framework where clients collaboratively train a global model without sharing their raw data. FedAvg is a popular algorithm for federated learning, but it often suffers from slow convergence due to the heterogeneity of local datasets and anisotropy in the parameter space. In this work, we formalize the central server optimization procedure through the lens of mirror descent and propose a novel framework, called FedDuA, which adaptively selects the global learning rate based on both inter-client and coordinate-wise heterogeneity in the local updates. We prove that our proposed doubly adaptive step-size rule is minimax optimal and provide a convergence analysis for convex objectives. Although the proposed method does not require additional communication or computational cost on clients, extensive numerical experiments show that our proposed framework outperforms baselines in various settings and is robust to the choice of hyperparameters.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-takakura26a, title = { FedDuA: Doubly Adaptive Federated Learning }, author = {Takakura, Shokichi and Liew, Seng Pei and Hasegawa, Satoshi}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {586--594}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/takakura26a/takakura26a.pdf}, url = {https://proceedings.mlr.press/v300/takakura26a.html}, abstract = { Federated learning is a distributed learning framework where clients collaboratively train a global model without sharing their raw data. FedAvg is a popular algorithm for federated learning, but it often suffers from slow convergence due to the heterogeneity of local datasets and anisotropy in the parameter space. In this work, we formalize the central server optimization procedure through the lens of mirror descent and propose a novel framework, called FedDuA, which adaptively selects the global learning rate based on both inter-client and coordinate-wise heterogeneity in the local updates. We prove that our proposed doubly adaptive step-size rule is minimax optimal and provide a convergence analysis for convex objectives. Although the proposed method does not require additional communication or computational cost on clients, extensive numerical experiments show that our proposed framework outperforms baselines in various settings and is robust to the choice of hyperparameters. } }
Endnote
%0 Conference Paper %T FedDuA: Doubly Adaptive Federated Learning %A Shokichi Takakura %A Seng Pei Liew %A Satoshi Hasegawa %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-takakura26a %I PMLR %P 586--594 %U https://proceedings.mlr.press/v300/takakura26a.html %V 300 %X Federated learning is a distributed learning framework where clients collaboratively train a global model without sharing their raw data. FedAvg is a popular algorithm for federated learning, but it often suffers from slow convergence due to the heterogeneity of local datasets and anisotropy in the parameter space. In this work, we formalize the central server optimization procedure through the lens of mirror descent and propose a novel framework, called FedDuA, which adaptively selects the global learning rate based on both inter-client and coordinate-wise heterogeneity in the local updates. We prove that our proposed doubly adaptive step-size rule is minimax optimal and provide a convergence analysis for convex objectives. Although the proposed method does not require additional communication or computational cost on clients, extensive numerical experiments show that our proposed framework outperforms baselines in various settings and is robust to the choice of hyperparameters.
APA
Takakura, S., Liew, S.P. & Hasegawa, S.. (2026). FedDuA: Doubly Adaptive Federated Learning . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:586-594 Available from https://proceedings.mlr.press/v300/takakura26a.html.

Related Material