Reconstructing Training Data from Model Gradient, Provably

Zihan Wang, Jason Lee, Qi Lei
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:6595-6612, 2023.

Abstract

Understanding when and how much a model gradient leaks information about the training sample is an important question in privacy. In this paper, we present a surprising result: Even without training or memorizing the data, we can fully reconstruct the training samples from a single gradient query at a randomly chosen parameter value. We prove the identifiability of the training data under mild assumptions: with shallow or deep neural networks and wide range of activation functions. We also present a statistically and computationally efficient algorithm based on tensor decomposition to reconstruct the training data. As a provable attack that reveals sensitive training data, our findings suggest potential severe threats to privacy, especially in federated learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-wang23g, title = {Reconstructing Training Data from Model Gradient, Provably}, author = {Wang, Zihan and Lee, Jason and Lei, Qi}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {6595--6612}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/wang23g/wang23g.pdf}, url = {https://proceedings.mlr.press/v206/wang23g.html}, abstract = {Understanding when and how much a model gradient leaks information about the training sample is an important question in privacy. In this paper, we present a surprising result: Even without training or memorizing the data, we can fully reconstruct the training samples from a single gradient query at a randomly chosen parameter value. We prove the identifiability of the training data under mild assumptions: with shallow or deep neural networks and wide range of activation functions. We also present a statistically and computationally efficient algorithm based on tensor decomposition to reconstruct the training data. As a provable attack that reveals sensitive training data, our findings suggest potential severe threats to privacy, especially in federated learning.} }
Endnote
%0 Conference Paper %T Reconstructing Training Data from Model Gradient, Provably %A Zihan Wang %A Jason Lee %A Qi Lei %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-wang23g %I PMLR %P 6595--6612 %U https://proceedings.mlr.press/v206/wang23g.html %V 206 %X Understanding when and how much a model gradient leaks information about the training sample is an important question in privacy. In this paper, we present a surprising result: Even without training or memorizing the data, we can fully reconstruct the training samples from a single gradient query at a randomly chosen parameter value. We prove the identifiability of the training data under mild assumptions: with shallow or deep neural networks and wide range of activation functions. We also present a statistically and computationally efficient algorithm based on tensor decomposition to reconstruct the training data. As a provable attack that reveals sensitive training data, our findings suggest potential severe threats to privacy, especially in federated learning.
APA
Wang, Z., Lee, J. & Lei, Q.. (2023). Reconstructing Training Data from Model Gradient, Provably. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:6595-6612 Available from https://proceedings.mlr.press/v206/wang23g.html.

Related Material