White-box vs Black-box: Bayes Optimal Strategies for Membership Inference

Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, Herve Jegou
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:5558-5567, 2019.

Abstract

Membership inference determines, given a sample and trained parameters of a machine learning model, whether the sample was part of the training set. In this paper, we derive the optimal strategy for membership inference with a few assumptions on the distribution of the parameters. We show that optimal attacks only depend on the loss function, and thus black-box attacks are as good as white-box attacks. As the optimal strategy is not tractable, we provide approximations of it leading to several inference methods, and show that existing membership inference methods are coarser approximations of this optimal strategy. Our membership attacks outperform the state of the art in various settings, ranging from a simple logistic regression to more complex architectures and datasets, such as ResNet-101 and Imagenet.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-sablayrolles19a, title = {White-box vs Black-box: {B}ayes Optimal Strategies for Membership Inference}, author = {Sablayrolles, Alexandre and Douze, Matthijs and Schmid, Cordelia and Ollivier, Yann and Jegou, Herve}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {5558--5567}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/sablayrolles19a/sablayrolles19a.pdf}, url = {https://proceedings.mlr.press/v97/sablayrolles19a.html}, abstract = {Membership inference determines, given a sample and trained parameters of a machine learning model, whether the sample was part of the training set. In this paper, we derive the optimal strategy for membership inference with a few assumptions on the distribution of the parameters. We show that optimal attacks only depend on the loss function, and thus black-box attacks are as good as white-box attacks. As the optimal strategy is not tractable, we provide approximations of it leading to several inference methods, and show that existing membership inference methods are coarser approximations of this optimal strategy. Our membership attacks outperform the state of the art in various settings, ranging from a simple logistic regression to more complex architectures and datasets, such as ResNet-101 and Imagenet.} }
Endnote
%0 Conference Paper %T White-box vs Black-box: Bayes Optimal Strategies for Membership Inference %A Alexandre Sablayrolles %A Matthijs Douze %A Cordelia Schmid %A Yann Ollivier %A Herve Jegou %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-sablayrolles19a %I PMLR %P 5558--5567 %U https://proceedings.mlr.press/v97/sablayrolles19a.html %V 97 %X Membership inference determines, given a sample and trained parameters of a machine learning model, whether the sample was part of the training set. In this paper, we derive the optimal strategy for membership inference with a few assumptions on the distribution of the parameters. We show that optimal attacks only depend on the loss function, and thus black-box attacks are as good as white-box attacks. As the optimal strategy is not tractable, we provide approximations of it leading to several inference methods, and show that existing membership inference methods are coarser approximations of this optimal strategy. Our membership attacks outperform the state of the art in various settings, ranging from a simple logistic regression to more complex architectures and datasets, such as ResNet-101 and Imagenet.
APA
Sablayrolles, A., Douze, M., Schmid, C., Ollivier, Y. & Jegou, H.. (2019). White-box vs Black-box: Bayes Optimal Strategies for Membership Inference. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:5558-5567 Available from https://proceedings.mlr.press/v97/sablayrolles19a.html.

Related Material