Differentially Private Learning of Undirected Graphical Models Using Collective Graphical Models

Garrett Bernstein, Ryan McKenna, Tao Sun, Daniel Sheldon, Michael Hay, Gerome Miklau
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:478-487, 2017.

Abstract

We investigate the problem of learning discrete graphical models in a differentially private way. Approaches to this problem range from privileged algorithms that conduct learning completely behind the privacy barrier to schemes that release private summary statistics paired with algorithms to learn parameters from those statistics. We show that the approach of releasing noisy sufficient statistics using the Laplace mechanism achieves a good trade-off between privacy, utility, and practicality. A naive learning algorithm that uses the noisy sufficient statistics “as is” outperforms general-purpose differentially private learning algorithms. However, it has three limitations: it ignores knowledge about the data generating process, rests on uncertain theoretical foundations, and exhibits certain pathologies. We develop a more principled approach that applies the formalism of collective graphical models to perform inference over the true sufficient statistics within an expectation-maximization framework. We show that this learns better models than competing approaches on both synthetic data and on real human mobility data used as a case study.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-bernstein17a, title = {Differentially Private Learning of Undirected Graphical Models Using Collective Graphical Models}, author = {Garrett Bernstein and Ryan McKenna and Tao Sun and Daniel Sheldon and Michael Hay and Gerome Miklau}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {478--487}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/bernstein17a/bernstein17a.pdf}, url = {https://proceedings.mlr.press/v70/bernstein17a.html}, abstract = {We investigate the problem of learning discrete graphical models in a differentially private way. Approaches to this problem range from privileged algorithms that conduct learning completely behind the privacy barrier to schemes that release private summary statistics paired with algorithms to learn parameters from those statistics. We show that the approach of releasing noisy sufficient statistics using the Laplace mechanism achieves a good trade-off between privacy, utility, and practicality. A naive learning algorithm that uses the noisy sufficient statistics “as is” outperforms general-purpose differentially private learning algorithms. However, it has three limitations: it ignores knowledge about the data generating process, rests on uncertain theoretical foundations, and exhibits certain pathologies. We develop a more principled approach that applies the formalism of collective graphical models to perform inference over the true sufficient statistics within an expectation-maximization framework. We show that this learns better models than competing approaches on both synthetic data and on real human mobility data used as a case study.} }
Endnote
%0 Conference Paper %T Differentially Private Learning of Undirected Graphical Models Using Collective Graphical Models %A Garrett Bernstein %A Ryan McKenna %A Tao Sun %A Daniel Sheldon %A Michael Hay %A Gerome Miklau %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-bernstein17a %I PMLR %P 478--487 %U https://proceedings.mlr.press/v70/bernstein17a.html %V 70 %X We investigate the problem of learning discrete graphical models in a differentially private way. Approaches to this problem range from privileged algorithms that conduct learning completely behind the privacy barrier to schemes that release private summary statistics paired with algorithms to learn parameters from those statistics. We show that the approach of releasing noisy sufficient statistics using the Laplace mechanism achieves a good trade-off between privacy, utility, and practicality. A naive learning algorithm that uses the noisy sufficient statistics “as is” outperforms general-purpose differentially private learning algorithms. However, it has three limitations: it ignores knowledge about the data generating process, rests on uncertain theoretical foundations, and exhibits certain pathologies. We develop a more principled approach that applies the formalism of collective graphical models to perform inference over the true sufficient statistics within an expectation-maximization framework. We show that this learns better models than competing approaches on both synthetic data and on real human mobility data used as a case study.
APA
Bernstein, G., McKenna, R., Sun, T., Sheldon, D., Hay, M. & Miklau, G.. (2017). Differentially Private Learning of Undirected Graphical Models Using Collective Graphical Models. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:478-487 Available from https://proceedings.mlr.press/v70/bernstein17a.html.

Related Material