FedScar: Correcting Geometric Bias for Flatness-Consistent Federated Learning

Jianfeng Lu, Yuzhao Xiang, Yue Chen, Gang Li, Shuqin Cao, Guanghui Wen
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:81796-81823, 2026.

Abstract

Federated Learning (FL) often suffers from degraded generalization under statistical heterogeneity, where client updates systematically deviate from the global objective. While recent Sharpness-Aware Minimization (SAM) methods promote locally flat solutions, they implicitly assume that local flatness transfers to the global model, which generally does not hold under heterogeneous data distributions. This mismatch gives rise to a flatness discrepancy induced by misaligned loss landscapes. To address this issue, we propose FedScar, a federated optimization framework that explicitly corrects heterogeneityinduced geometric inconsistency. FedScar maintains a history-accumulated geometric bias to capture persistent curvature skew across clients, and employs a variance-aware injection mechanism to steer local updates toward regions that are flat with respect to the global objective. We provide a theoretical interpretation of FedScar as a Split-Dual ADMM formulation, which jointly enforces parameter consensus and geometric alignment. Extensive experiments under severe heterogeneity demonstrate that FedScar consistently reduces flatness discrepancy and improves generalization over state-of-the-art methods, without incurring additional communication overhead. The source code is available at https://github.com/Ultraman6/FedScar.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-lu26ap, title = {{F}ed{S}car: Correcting Geometric Bias for Flatness-Consistent Federated Learning}, author = {Lu, Jianfeng and Xiang, Yuzhao and Chen, Yue and Li, Gang and Cao, Shuqin and Wen, Guanghui}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {81796--81823}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/lu26ap/lu26ap.pdf}, url = {https://proceedings.mlr.press/v306/lu26ap.html}, abstract = {Federated Learning (FL) often suffers from degraded generalization under statistical heterogeneity, where client updates systematically deviate from the global objective. While recent Sharpness-Aware Minimization (SAM) methods promote locally flat solutions, they implicitly assume that local flatness transfers to the global model, which generally does not hold under heterogeneous data distributions. This mismatch gives rise to a flatness discrepancy induced by misaligned loss landscapes. To address this issue, we propose FedScar, a federated optimization framework that explicitly corrects heterogeneityinduced geometric inconsistency. FedScar maintains a history-accumulated geometric bias to capture persistent curvature skew across clients, and employs a variance-aware injection mechanism to steer local updates toward regions that are flat with respect to the global objective. We provide a theoretical interpretation of FedScar as a Split-Dual ADMM formulation, which jointly enforces parameter consensus and geometric alignment. Extensive experiments under severe heterogeneity demonstrate that FedScar consistently reduces flatness discrepancy and improves generalization over state-of-the-art methods, without incurring additional communication overhead. The source code is available at https://github.com/Ultraman6/FedScar.} }
Endnote
%0 Conference Paper %T FedScar: Correcting Geometric Bias for Flatness-Consistent Federated Learning %A Jianfeng Lu %A Yuzhao Xiang %A Yue Chen %A Gang Li %A Shuqin Cao %A Guanghui Wen %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-lu26ap %I PMLR %P 81796--81823 %U https://proceedings.mlr.press/v306/lu26ap.html %V 306 %X Federated Learning (FL) often suffers from degraded generalization under statistical heterogeneity, where client updates systematically deviate from the global objective. While recent Sharpness-Aware Minimization (SAM) methods promote locally flat solutions, they implicitly assume that local flatness transfers to the global model, which generally does not hold under heterogeneous data distributions. This mismatch gives rise to a flatness discrepancy induced by misaligned loss landscapes. To address this issue, we propose FedScar, a federated optimization framework that explicitly corrects heterogeneityinduced geometric inconsistency. FedScar maintains a history-accumulated geometric bias to capture persistent curvature skew across clients, and employs a variance-aware injection mechanism to steer local updates toward regions that are flat with respect to the global objective. We provide a theoretical interpretation of FedScar as a Split-Dual ADMM formulation, which jointly enforces parameter consensus and geometric alignment. Extensive experiments under severe heterogeneity demonstrate that FedScar consistently reduces flatness discrepancy and improves generalization over state-of-the-art methods, without incurring additional communication overhead. The source code is available at https://github.com/Ultraman6/FedScar.
APA
Lu, J., Xiang, Y., Chen, Y., Li, G., Cao, S. & Wen, G.. (2026). FedScar: Correcting Geometric Bias for Flatness-Consistent Federated Learning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:81796-81823 Available from https://proceedings.mlr.press/v306/lu26ap.html.

Related Material