FedSteer: Taming Extreme Gradient Staleness in Federated Learning with Corrective Projections and Caching

Haoran Zhang, Cainã Figueiredo Pereira, Marie Siew, Xutong Liu, Carlee Joe-Wong, Rachid El-Azouzi
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:8090-8114, 2026.

Abstract

Federated learning (FL) is often subject to aggregation variance if clients do not consistently participate in training rounds. While reusing stale model updates from inactive clients is a common technique to reduce this variance, we find that with skewed client participation, the resulting update staleness can become severe enough to destabilize training. To remedy this, we propose FedSteer, a novel method that constructs a gradient subspace from a cache of recent client gradients to serve as a low-dimensional representation of the current optimization landscape. FedSteer projects an active client’s true gradient onto this subspace to find a set of optimal coordinates. For an inactive client, FedSteer reuses these coordinates with the now-evolved subspace drifted by other active clients. This process effectively “steers” outdated gradients toward the current global objective. This is complemented by a selective caching strategy that identifies a representative client subset to form the subspace, reducing server memory. Experiments demonstrate that FedSteer significantly outperforms baselines, preventing performance collapse in challenging scenarios while delivering accuracy gains of over 7% in others.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-zhang26d, title = {FedSteer: Taming Extreme Gradient Staleness in Federated Learning with Corrective Projections and Caching}, author = {Zhang, Haoran and Pereira, Cain{\~a} Figueiredo and Siew, Marie and Liu, Xutong and Joe-Wong, Carlee and El-Azouzi, Rachid}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {8090--8114}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/zhang26d/zhang26d.pdf}, url = {https://proceedings.mlr.press/v337/zhang26d.html}, abstract = {Federated learning (FL) is often subject to aggregation variance if clients do not consistently participate in training rounds. While reusing stale model updates from inactive clients is a common technique to reduce this variance, we find that with skewed client participation, the resulting update staleness can become severe enough to destabilize training. To remedy this, we propose FedSteer, a novel method that constructs a gradient subspace from a cache of recent client gradients to serve as a low-dimensional representation of the current optimization landscape. FedSteer projects an active client’s true gradient onto this subspace to find a set of optimal coordinates. For an inactive client, FedSteer reuses these coordinates with the now-evolved subspace drifted by other active clients. This process effectively “steers” outdated gradients toward the current global objective. This is complemented by a selective caching strategy that identifies a representative client subset to form the subspace, reducing server memory. Experiments demonstrate that FedSteer significantly outperforms baselines, preventing performance collapse in challenging scenarios while delivering accuracy gains of over 7% in others.} }
Endnote
%0 Conference Paper %T FedSteer: Taming Extreme Gradient Staleness in Federated Learning with Corrective Projections and Caching %A Haoran Zhang %A Cainã Figueiredo Pereira %A Marie Siew %A Xutong Liu %A Carlee Joe-Wong %A Rachid El-Azouzi %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-zhang26d %I PMLR %P 8090--8114 %U https://proceedings.mlr.press/v337/zhang26d.html %V 337 %X Federated learning (FL) is often subject to aggregation variance if clients do not consistently participate in training rounds. While reusing stale model updates from inactive clients is a common technique to reduce this variance, we find that with skewed client participation, the resulting update staleness can become severe enough to destabilize training. To remedy this, we propose FedSteer, a novel method that constructs a gradient subspace from a cache of recent client gradients to serve as a low-dimensional representation of the current optimization landscape. FedSteer projects an active client’s true gradient onto this subspace to find a set of optimal coordinates. For an inactive client, FedSteer reuses these coordinates with the now-evolved subspace drifted by other active clients. This process effectively “steers” outdated gradients toward the current global objective. This is complemented by a selective caching strategy that identifies a representative client subset to form the subspace, reducing server memory. Experiments demonstrate that FedSteer significantly outperforms baselines, preventing performance collapse in challenging scenarios while delivering accuracy gains of over 7% in others.
APA
Zhang, H., Pereira, C.F., Siew, M., Liu, X., Joe-Wong, C. & El-Azouzi, R.. (2026). FedSteer: Taming Extreme Gradient Staleness in Federated Learning with Corrective Projections and Caching. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:8090-8114 Available from https://proceedings.mlr.press/v337/zhang26d.html.

Related Material