One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory

Yuhang Li, Tong Liu, Yangguang Cui, Ming Hu, Xiaoqiang Li
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:35317-35341, 2025.

Abstract

Federated learning (FL) presents a promising strategy for distributed and privacy-preserving learning, yet struggles with performance issues in the presence of heterogeneous data distributions. Recently, a series of works based on sharpness-aware minimization (SAM) have emerged to improve local learning generality, proving to be effective in mitigating data heterogeneity effects. However, most SAM-based methods do not directly consider the global objective and require two backward pass per iteration, resulting in diminished effectiveness. To overcome these two bottlenecks, we leverage the global model trajectory to directly measure sharpness for the global objective, requiring only a single backward pass. We further propose a novel and general algorithm FedGMT to overcome data heterogeneity and the pitfalls of previous SAM-based methods. We analyze the convergence of FedGMT and conduct extensive experiments on visual and text datasets in a variety of scenarios, demonstrating that FedGMT achieves competitive accuracy with state-of-the-art FL methods while minimizing computation and communication overhead. Code is available at https://github.com/harrylee999/FL-SAM.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-li25bd, title = {One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory}, author = {Li, Yuhang and Liu, Tong and Cui, Yangguang and Hu, Ming and Li, Xiaoqiang}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {35317--35341}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/li25bd/li25bd.pdf}, url = {https://proceedings.mlr.press/v267/li25bd.html}, abstract = {Federated learning (FL) presents a promising strategy for distributed and privacy-preserving learning, yet struggles with performance issues in the presence of heterogeneous data distributions. Recently, a series of works based on sharpness-aware minimization (SAM) have emerged to improve local learning generality, proving to be effective in mitigating data heterogeneity effects. However, most SAM-based methods do not directly consider the global objective and require two backward pass per iteration, resulting in diminished effectiveness. To overcome these two bottlenecks, we leverage the global model trajectory to directly measure sharpness for the global objective, requiring only a single backward pass. We further propose a novel and general algorithm FedGMT to overcome data heterogeneity and the pitfalls of previous SAM-based methods. We analyze the convergence of FedGMT and conduct extensive experiments on visual and text datasets in a variety of scenarios, demonstrating that FedGMT achieves competitive accuracy with state-of-the-art FL methods while minimizing computation and communication overhead. Code is available at https://github.com/harrylee999/FL-SAM.} }
Endnote
%0 Conference Paper %T One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory %A Yuhang Li %A Tong Liu %A Yangguang Cui %A Ming Hu %A Xiaoqiang Li %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-li25bd %I PMLR %P 35317--35341 %U https://proceedings.mlr.press/v267/li25bd.html %V 267 %X Federated learning (FL) presents a promising strategy for distributed and privacy-preserving learning, yet struggles with performance issues in the presence of heterogeneous data distributions. Recently, a series of works based on sharpness-aware minimization (SAM) have emerged to improve local learning generality, proving to be effective in mitigating data heterogeneity effects. However, most SAM-based methods do not directly consider the global objective and require two backward pass per iteration, resulting in diminished effectiveness. To overcome these two bottlenecks, we leverage the global model trajectory to directly measure sharpness for the global objective, requiring only a single backward pass. We further propose a novel and general algorithm FedGMT to overcome data heterogeneity and the pitfalls of previous SAM-based methods. We analyze the convergence of FedGMT and conduct extensive experiments on visual and text datasets in a variety of scenarios, demonstrating that FedGMT achieves competitive accuracy with state-of-the-art FL methods while minimizing computation and communication overhead. Code is available at https://github.com/harrylee999/FL-SAM.
APA
Li, Y., Liu, T., Cui, Y., Hu, M. & Li, X.. (2025). One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:35317-35341 Available from https://proceedings.mlr.press/v267/li25bd.html.

Related Material