Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning

Alberto Bietti, Chen-Yu Wei, Miroslav Dudik, John Langford, Steven Wu
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:1945-1962, 2022.

Abstract

Large-scale machine learning systems often involve data distributed across a collection of users. Federated learning algorithms leverage this structure by communicating model updates to a central server, rather than entire datasets. In this paper, we study stochastic optimization algorithms for a personalized federated learning setting involving local and global models subject to user-level (joint) differential privacy. While learning a private global model induces a cost of privacy, local learning is perfectly private. We provide generalization guarantees showing that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-bietti22a, title = {Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning}, author = {Bietti, Alberto and Wei, Chen-Yu and Dudik, Miroslav and Langford, John and Wu, Steven}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {1945--1962}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/bietti22a/bietti22a.pdf}, url = {https://proceedings.mlr.press/v162/bietti22a.html}, abstract = {Large-scale machine learning systems often involve data distributed across a collection of users. Federated learning algorithms leverage this structure by communicating model updates to a central server, rather than entire datasets. In this paper, we study stochastic optimization algorithms for a personalized federated learning setting involving local and global models subject to user-level (joint) differential privacy. While learning a private global model induces a cost of privacy, local learning is perfectly private. We provide generalization guarantees showing that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.} }
Endnote
%0 Conference Paper %T Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning %A Alberto Bietti %A Chen-Yu Wei %A Miroslav Dudik %A John Langford %A Steven Wu %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-bietti22a %I PMLR %P 1945--1962 %U https://proceedings.mlr.press/v162/bietti22a.html %V 162 %X Large-scale machine learning systems often involve data distributed across a collection of users. Federated learning algorithms leverage this structure by communicating model updates to a central server, rather than entire datasets. In this paper, we study stochastic optimization algorithms for a personalized federated learning setting involving local and global models subject to user-level (joint) differential privacy. While learning a private global model induces a cost of privacy, local learning is perfectly private. We provide generalization guarantees showing that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.
APA
Bietti, A., Wei, C., Dudik, M., Langford, J. & Wu, S.. (2022). Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:1945-1962 Available from https://proceedings.mlr.press/v162/bietti22a.html.

Related Material