Differentially Private Sum-Product Networks

Xenia Heilmann, Mattia Cerrato, Ernst Althaus
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:18155-18173, 2024.

Abstract

Differentially private ML approaches seek to learn models which may be publicly released while guaranteeing that the input data is kept private. One issue with this construction is that further model releases based on the same training data (e.g. for a new task) incur a further privacy budget cost. Privacy-preserving synthetic data generation is one possible solution to this conundrum. However, models trained on synthetic private data struggle to approach the performance of private, ad-hoc models. In this paper, we present a novel method based on sum-product networks that is able to perform both privacy-preserving classification and privacy-preserving data generation with a single model. To the best of our knowledge, ours is the first approach that provides both discriminative and generative capabilities to differentially private ML. We show that our approach outperforms the state of the art in terms of stability (i.e. number of training runs required for convergence) and utility of the generated data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-heilmann24a, title = {Differentially Private Sum-Product Networks}, author = {Heilmann, Xenia and Cerrato, Mattia and Althaus, Ernst}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {18155--18173}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/heilmann24a/heilmann24a.pdf}, url = {https://proceedings.mlr.press/v235/heilmann24a.html}, abstract = {Differentially private ML approaches seek to learn models which may be publicly released while guaranteeing that the input data is kept private. One issue with this construction is that further model releases based on the same training data (e.g. for a new task) incur a further privacy budget cost. Privacy-preserving synthetic data generation is one possible solution to this conundrum. However, models trained on synthetic private data struggle to approach the performance of private, ad-hoc models. In this paper, we present a novel method based on sum-product networks that is able to perform both privacy-preserving classification and privacy-preserving data generation with a single model. To the best of our knowledge, ours is the first approach that provides both discriminative and generative capabilities to differentially private ML. We show that our approach outperforms the state of the art in terms of stability (i.e. number of training runs required for convergence) and utility of the generated data.} }
Endnote
%0 Conference Paper %T Differentially Private Sum-Product Networks %A Xenia Heilmann %A Mattia Cerrato %A Ernst Althaus %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-heilmann24a %I PMLR %P 18155--18173 %U https://proceedings.mlr.press/v235/heilmann24a.html %V 235 %X Differentially private ML approaches seek to learn models which may be publicly released while guaranteeing that the input data is kept private. One issue with this construction is that further model releases based on the same training data (e.g. for a new task) incur a further privacy budget cost. Privacy-preserving synthetic data generation is one possible solution to this conundrum. However, models trained on synthetic private data struggle to approach the performance of private, ad-hoc models. In this paper, we present a novel method based on sum-product networks that is able to perform both privacy-preserving classification and privacy-preserving data generation with a single model. To the best of our knowledge, ours is the first approach that provides both discriminative and generative capabilities to differentially private ML. We show that our approach outperforms the state of the art in terms of stability (i.e. number of training runs required for convergence) and utility of the generated data.
APA
Heilmann, X., Cerrato, M. & Althaus, E.. (2024). Differentially Private Sum-Product Networks. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:18155-18173 Available from https://proceedings.mlr.press/v235/heilmann24a.html.

Related Material