Low-Cost High-Power Membership Inference Attacks

Sajjad Zarifzadeh, Philippe Liu, Reza Shokri
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:58244-58282, 2024.

Abstract

Membership inference attacks aim to detect if a particular data point was used in training a model. We design a novel statistical test to perform robust membership inference attacks (RMIA) with low computational overhead. We achieve this by a fine-grained modeling of the null hypothesis in our likelihood ratio tests, and effectively leveraging both reference models and reference population data samples. RMIA has superior test power compared with prior methods, throughout the TPR-FPR curve (even at extremely low FPR, as low as 0). Under computational constraints, where only a limited number of pre-trained reference models (as few as 1) are available, and also when we vary other elements of the attack (e.g., data distribution), our method performs exceptionally well, unlike prior attacks that approach random guessing. RMIA lays the groundwork for practical yet accurate data privacy risk assessment in machine learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-zarifzadeh24a, title = {Low-Cost High-Power Membership Inference Attacks}, author = {Zarifzadeh, Sajjad and Liu, Philippe and Shokri, Reza}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {58244--58282}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/zarifzadeh24a/zarifzadeh24a.pdf}, url = {https://proceedings.mlr.press/v235/zarifzadeh24a.html}, abstract = {Membership inference attacks aim to detect if a particular data point was used in training a model. We design a novel statistical test to perform robust membership inference attacks (RMIA) with low computational overhead. We achieve this by a fine-grained modeling of the null hypothesis in our likelihood ratio tests, and effectively leveraging both reference models and reference population data samples. RMIA has superior test power compared with prior methods, throughout the TPR-FPR curve (even at extremely low FPR, as low as 0). Under computational constraints, where only a limited number of pre-trained reference models (as few as 1) are available, and also when we vary other elements of the attack (e.g., data distribution), our method performs exceptionally well, unlike prior attacks that approach random guessing. RMIA lays the groundwork for practical yet accurate data privacy risk assessment in machine learning.} }
Endnote
%0 Conference Paper %T Low-Cost High-Power Membership Inference Attacks %A Sajjad Zarifzadeh %A Philippe Liu %A Reza Shokri %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-zarifzadeh24a %I PMLR %P 58244--58282 %U https://proceedings.mlr.press/v235/zarifzadeh24a.html %V 235 %X Membership inference attacks aim to detect if a particular data point was used in training a model. We design a novel statistical test to perform robust membership inference attacks (RMIA) with low computational overhead. We achieve this by a fine-grained modeling of the null hypothesis in our likelihood ratio tests, and effectively leveraging both reference models and reference population data samples. RMIA has superior test power compared with prior methods, throughout the TPR-FPR curve (even at extremely low FPR, as low as 0). Under computational constraints, where only a limited number of pre-trained reference models (as few as 1) are available, and also when we vary other elements of the attack (e.g., data distribution), our method performs exceptionally well, unlike prior attacks that approach random guessing. RMIA lays the groundwork for practical yet accurate data privacy risk assessment in machine learning.
APA
Zarifzadeh, S., Liu, P. & Shokri, R.. (2024). Low-Cost High-Power Membership Inference Attacks. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:58244-58282 Available from https://proceedings.mlr.press/v235/zarifzadeh24a.html.

Related Material