Characterizing Internal Evasion Attacks in Federated Learning

Taejin Kim, Shubhranshu Singh, Nikhil Madaan, Carlee Joe-Wong
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:907-921, 2023.

Abstract

Federated learning allows for clients in a distributed system to jointly train a machine learning model. However, clients’ models are vulnerable to attacks during the training and testing phases. In this paper, we address the issue of adversarial clients performing “internal evasion attacks”: crafting evasion attacks at test time to deceive other clients. For example, adversaries may aim to deceive spam filters and recommendation systems trained with federated learning for monetary gain. The adversarial clients have extensive information about the victim model in a federated learning setting, as weight information is shared amongst clients. We are the first to characterize the transferability of such internal evasion attacks for different learning methods and analyze the trade-off between model accuracy and robustness depending on the degree of similarities in client data. We show that adversarial training defenses in the federated learning setting only display limited improvements against internal attacks. However, combining adversarial training with personalized federated learning frameworks increases relative internal attack robustness by 60$%$ compared to federated adversarial training and performs well under limited system resources.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-kim23a, title = {Characterizing Internal Evasion Attacks in Federated Learning}, author = {Kim, Taejin and Singh, Shubhranshu and Madaan, Nikhil and Joe-Wong, Carlee}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {907--921}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/kim23a/kim23a.pdf}, url = {https://proceedings.mlr.press/v206/kim23a.html}, abstract = {Federated learning allows for clients in a distributed system to jointly train a machine learning model. However, clients’ models are vulnerable to attacks during the training and testing phases. In this paper, we address the issue of adversarial clients performing “internal evasion attacks”: crafting evasion attacks at test time to deceive other clients. For example, adversaries may aim to deceive spam filters and recommendation systems trained with federated learning for monetary gain. The adversarial clients have extensive information about the victim model in a federated learning setting, as weight information is shared amongst clients. We are the first to characterize the transferability of such internal evasion attacks for different learning methods and analyze the trade-off between model accuracy and robustness depending on the degree of similarities in client data. We show that adversarial training defenses in the federated learning setting only display limited improvements against internal attacks. However, combining adversarial training with personalized federated learning frameworks increases relative internal attack robustness by 60$%$ compared to federated adversarial training and performs well under limited system resources.} }
Endnote
%0 Conference Paper %T Characterizing Internal Evasion Attacks in Federated Learning %A Taejin Kim %A Shubhranshu Singh %A Nikhil Madaan %A Carlee Joe-Wong %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-kim23a %I PMLR %P 907--921 %U https://proceedings.mlr.press/v206/kim23a.html %V 206 %X Federated learning allows for clients in a distributed system to jointly train a machine learning model. However, clients’ models are vulnerable to attacks during the training and testing phases. In this paper, we address the issue of adversarial clients performing “internal evasion attacks”: crafting evasion attacks at test time to deceive other clients. For example, adversaries may aim to deceive spam filters and recommendation systems trained with federated learning for monetary gain. The adversarial clients have extensive information about the victim model in a federated learning setting, as weight information is shared amongst clients. We are the first to characterize the transferability of such internal evasion attacks for different learning methods and analyze the trade-off between model accuracy and robustness depending on the degree of similarities in client data. We show that adversarial training defenses in the federated learning setting only display limited improvements against internal attacks. However, combining adversarial training with personalized federated learning frameworks increases relative internal attack robustness by 60$%$ compared to federated adversarial training and performs well under limited system resources.
APA
Kim, T., Singh, S., Madaan, N. & Joe-Wong, C.. (2023). Characterizing Internal Evasion Attacks in Federated Learning. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:907-921 Available from https://proceedings.mlr.press/v206/kim23a.html.

Related Material