Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix

Maximilian Lam, Gu-Yeon Wei, David Brooks, Vijay Janapa Reddi, Michael Mitzenmacher
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:5959-5968, 2021.

Abstract

We show that aggregated model updates in federated learning may be insecure. An untrusted central server may disaggregate user updates from sums of updates across participants given repeated observations, enabling the server to recover privileged information about individual users’ private training data via traditional gradient inference attacks. Our method revolves around reconstructing participant information (e.g: which rounds of training users participated in) from aggregated model updates by leveraging summary information from device analytics commonly used to monitor, debug, and manage federated learning systems. Our attack is parallelizable and we successfully disaggregate user updates on settings with up to thousands of participants. We quantitatively and qualitatively demonstrate significant improvements in the capability of various inference attacks on the disaggregated updates. Our attack enables the attribution of learned properties to individual users, violating anonymity, and shows that a determined central server may undermine the secure aggregation protocol to break individual users’ data privacy in federated learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-lam21b, title = {Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix}, author = {Lam, Maximilian and Wei, Gu-Yeon and Brooks, David and Reddi, Vijay Janapa and Mitzenmacher, Michael}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {5959--5968}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/lam21b/lam21b.pdf}, url = {https://proceedings.mlr.press/v139/lam21b.html}, abstract = {We show that aggregated model updates in federated learning may be insecure. An untrusted central server may disaggregate user updates from sums of updates across participants given repeated observations, enabling the server to recover privileged information about individual users’ private training data via traditional gradient inference attacks. Our method revolves around reconstructing participant information (e.g: which rounds of training users participated in) from aggregated model updates by leveraging summary information from device analytics commonly used to monitor, debug, and manage federated learning systems. Our attack is parallelizable and we successfully disaggregate user updates on settings with up to thousands of participants. We quantitatively and qualitatively demonstrate significant improvements in the capability of various inference attacks on the disaggregated updates. Our attack enables the attribution of learned properties to individual users, violating anonymity, and shows that a determined central server may undermine the secure aggregation protocol to break individual users’ data privacy in federated learning.} }
Endnote
%0 Conference Paper %T Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix %A Maximilian Lam %A Gu-Yeon Wei %A David Brooks %A Vijay Janapa Reddi %A Michael Mitzenmacher %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-lam21b %I PMLR %P 5959--5968 %U https://proceedings.mlr.press/v139/lam21b.html %V 139 %X We show that aggregated model updates in federated learning may be insecure. An untrusted central server may disaggregate user updates from sums of updates across participants given repeated observations, enabling the server to recover privileged information about individual users’ private training data via traditional gradient inference attacks. Our method revolves around reconstructing participant information (e.g: which rounds of training users participated in) from aggregated model updates by leveraging summary information from device analytics commonly used to monitor, debug, and manage federated learning systems. Our attack is parallelizable and we successfully disaggregate user updates on settings with up to thousands of participants. We quantitatively and qualitatively demonstrate significant improvements in the capability of various inference attacks on the disaggregated updates. Our attack enables the attribution of learned properties to individual users, violating anonymity, and shows that a determined central server may undermine the secure aggregation protocol to break individual users’ data privacy in federated learning.
APA
Lam, M., Wei, G., Brooks, D., Reddi, V.J. & Mitzenmacher, M.. (2021). Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:5959-5968 Available from https://proceedings.mlr.press/v139/lam21b.html.

Related Material