BOBA: Byzantine-Robust Federated Learning with Label Skewness

Wenxuan Bao, Jun Wu, Jingrui He
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:892-900, 2024.

Abstract

In federated learning, most existing robust aggregation rules (AGRs) combat Byzantine attacks in the IID setting, where client data is assumed to be independent and identically distributed. In this paper, we address label skewness, a more realistic and challenging non-IID setting, where each client only has access to a few classes of data. In this setting, state-of-the-art AGRs suffer from selection bias, leading to significant performance drop for particular classes; they are also more vulnerable to Byzantine attacks due to the increased variation among gradients of honest clients. To address these limitations, we propose an efficient two-stage method named BOBA. Theoretically, we prove the convergence of BOBA with an error of the optimal order. Our empirical evaluations demonstrate BOBA’s superior unbiasedness and robustness across diverse models and datasets when compared to various baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-bao24a, title = { {BOBA}: Byzantine-Robust Federated Learning with Label Skewness }, author = {Bao, Wenxuan and Wu, Jun and He, Jingrui}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {892--900}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/bao24a/bao24a.pdf}, url = {https://proceedings.mlr.press/v238/bao24a.html}, abstract = { In federated learning, most existing robust aggregation rules (AGRs) combat Byzantine attacks in the IID setting, where client data is assumed to be independent and identically distributed. In this paper, we address label skewness, a more realistic and challenging non-IID setting, where each client only has access to a few classes of data. In this setting, state-of-the-art AGRs suffer from selection bias, leading to significant performance drop for particular classes; they are also more vulnerable to Byzantine attacks due to the increased variation among gradients of honest clients. To address these limitations, we propose an efficient two-stage method named BOBA. Theoretically, we prove the convergence of BOBA with an error of the optimal order. Our empirical evaluations demonstrate BOBA’s superior unbiasedness and robustness across diverse models and datasets when compared to various baselines. } }
Endnote
%0 Conference Paper %T BOBA: Byzantine-Robust Federated Learning with Label Skewness %A Wenxuan Bao %A Jun Wu %A Jingrui He %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-bao24a %I PMLR %P 892--900 %U https://proceedings.mlr.press/v238/bao24a.html %V 238 %X In federated learning, most existing robust aggregation rules (AGRs) combat Byzantine attacks in the IID setting, where client data is assumed to be independent and identically distributed. In this paper, we address label skewness, a more realistic and challenging non-IID setting, where each client only has access to a few classes of data. In this setting, state-of-the-art AGRs suffer from selection bias, leading to significant performance drop for particular classes; they are also more vulnerable to Byzantine attacks due to the increased variation among gradients of honest clients. To address these limitations, we propose an efficient two-stage method named BOBA. Theoretically, we prove the convergence of BOBA with an error of the optimal order. Our empirical evaluations demonstrate BOBA’s superior unbiasedness and robustness across diverse models and datasets when compared to various baselines.
APA
Bao, W., Wu, J. & He, J.. (2024). BOBA: Byzantine-Robust Federated Learning with Label Skewness . Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:892-900 Available from https://proceedings.mlr.press/v238/bao24a.html.

Related Material