Private Learning with Public Features

Walid Krichene, Nicolas E Mayoraz, Steffen Rendle, Shuang Song, Abhradeep Thakurta, Li Zhang
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:4150-4158, 2024.

Abstract

We study a class of private learning problems in which the data is a join of private and public features. This is often the case in private personalization tasks such as recommendation or ad prediction, in which features related to individuals are sensitive, while features related to items (the movies or songs to be recommended, or the ads to be shown to users) are publicly available and do not require protection. A natural question is whether private algorithms can achieve higher utility in the presence of public features. We give a positive answer for multi-encoder models where one of the encoders operates on public features. We develop new algorithms that take advantage of this separation by only protecting certain sufficient statistics (instead of adding noise to the gradient). This method has a guaranteed utility improvement for linear regression, and importantly, achieves the state of the art on two standard private recommendation benchmarks, demonstrating the importance of methods that adapt to the private-public feature separation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-krichene24a, title = {Private Learning with Public Features}, author = {Krichene, Walid and E Mayoraz, Nicolas and Rendle, Steffen and Song, Shuang and Thakurta, Abhradeep and Zhang, Li}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {4150--4158}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/krichene24a/krichene24a.pdf}, url = {https://proceedings.mlr.press/v238/krichene24a.html}, abstract = {We study a class of private learning problems in which the data is a join of private and public features. This is often the case in private personalization tasks such as recommendation or ad prediction, in which features related to individuals are sensitive, while features related to items (the movies or songs to be recommended, or the ads to be shown to users) are publicly available and do not require protection. A natural question is whether private algorithms can achieve higher utility in the presence of public features. We give a positive answer for multi-encoder models where one of the encoders operates on public features. We develop new algorithms that take advantage of this separation by only protecting certain sufficient statistics (instead of adding noise to the gradient). This method has a guaranteed utility improvement for linear regression, and importantly, achieves the state of the art on two standard private recommendation benchmarks, demonstrating the importance of methods that adapt to the private-public feature separation.} }
Endnote
%0 Conference Paper %T Private Learning with Public Features %A Walid Krichene %A Nicolas E Mayoraz %A Steffen Rendle %A Shuang Song %A Abhradeep Thakurta %A Li Zhang %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-krichene24a %I PMLR %P 4150--4158 %U https://proceedings.mlr.press/v238/krichene24a.html %V 238 %X We study a class of private learning problems in which the data is a join of private and public features. This is often the case in private personalization tasks such as recommendation or ad prediction, in which features related to individuals are sensitive, while features related to items (the movies or songs to be recommended, or the ads to be shown to users) are publicly available and do not require protection. A natural question is whether private algorithms can achieve higher utility in the presence of public features. We give a positive answer for multi-encoder models where one of the encoders operates on public features. We develop new algorithms that take advantage of this separation by only protecting certain sufficient statistics (instead of adding noise to the gradient). This method has a guaranteed utility improvement for linear regression, and importantly, achieves the state of the art on two standard private recommendation benchmarks, demonstrating the importance of methods that adapt to the private-public feature separation.
APA
Krichene, W., E Mayoraz, N., Rendle, S., Song, S., Thakurta, A. & Zhang, L.. (2024). Private Learning with Public Features. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:4150-4158 Available from https://proceedings.mlr.press/v238/krichene24a.html.

Related Material