Label Inference Attacks from Log-loss Scores

Abhinav Aggarwal, Shiva Kasiviswanathan, Zekun Xu, Oluwaseyi Feyisetan, Nathanael Teissier
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:120-129, 2021.

Abstract

Log-loss (also known as cross-entropy loss) metric is ubiquitously used across machine learning applications to assess the performance of classification algorithms. In this paper, we investigate the problem of inferring the labels of a dataset from single (or multiple) log-loss score(s), without any other access to the dataset. Surprisingly, we show that for any finite number of label classes, it is possible to accurately infer the labels of the dataset from the reported log-loss score of a single carefully constructed prediction vector if we allow arbitrary precision arithmetic. Additionally, we present label inference algorithms (attacks) that succeed even under addition of noise to the log-loss scores and under limited precision arithmetic. All our algorithms rely on ideas from number theory and combinatorics and require no model training. We run experimental simulations on some real datasets to demonstrate the ease of running these attacks in practice.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-aggarwal21a, title = {Label Inference Attacks from Log-loss Scores}, author = {Aggarwal, Abhinav and Kasiviswanathan, Shiva and Xu, Zekun and Feyisetan, Oluwaseyi and Teissier, Nathanael}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {120--129}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/aggarwal21a/aggarwal21a.pdf}, url = {https://proceedings.mlr.press/v139/aggarwal21a.html}, abstract = {Log-loss (also known as cross-entropy loss) metric is ubiquitously used across machine learning applications to assess the performance of classification algorithms. In this paper, we investigate the problem of inferring the labels of a dataset from single (or multiple) log-loss score(s), without any other access to the dataset. Surprisingly, we show that for any finite number of label classes, it is possible to accurately infer the labels of the dataset from the reported log-loss score of a single carefully constructed prediction vector if we allow arbitrary precision arithmetic. Additionally, we present label inference algorithms (attacks) that succeed even under addition of noise to the log-loss scores and under limited precision arithmetic. All our algorithms rely on ideas from number theory and combinatorics and require no model training. We run experimental simulations on some real datasets to demonstrate the ease of running these attacks in practice.} }
Endnote
%0 Conference Paper %T Label Inference Attacks from Log-loss Scores %A Abhinav Aggarwal %A Shiva Kasiviswanathan %A Zekun Xu %A Oluwaseyi Feyisetan %A Nathanael Teissier %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-aggarwal21a %I PMLR %P 120--129 %U https://proceedings.mlr.press/v139/aggarwal21a.html %V 139 %X Log-loss (also known as cross-entropy loss) metric is ubiquitously used across machine learning applications to assess the performance of classification algorithms. In this paper, we investigate the problem of inferring the labels of a dataset from single (or multiple) log-loss score(s), without any other access to the dataset. Surprisingly, we show that for any finite number of label classes, it is possible to accurately infer the labels of the dataset from the reported log-loss score of a single carefully constructed prediction vector if we allow arbitrary precision arithmetic. Additionally, we present label inference algorithms (attacks) that succeed even under addition of noise to the log-loss scores and under limited precision arithmetic. All our algorithms rely on ideas from number theory and combinatorics and require no model training. We run experimental simulations on some real datasets to demonstrate the ease of running these attacks in practice.
APA
Aggarwal, A., Kasiviswanathan, S., Xu, Z., Feyisetan, O. & Teissier, N.. (2021). Label Inference Attacks from Log-loss Scores. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:120-129 Available from https://proceedings.mlr.press/v139/aggarwal21a.html.

Related Material