On the Sample Complexity of Privately Learning Unbounded High-Dimensional Gaussians

Ishaq Aden-Ali, Hassan Ashtiani, Gautam Kamath
Proceedings of the 32nd International Conference on Algorithmic Learning Theory, PMLR 132:185-216, 2021.

Abstract

We provide sample complexity upper bounds for agnostically learning multivariate Gaussians under the constraint of approximate differential privacy. These are the first finite sample upper bounds for general Gaussians which do not impose restrictions on the parameters of the distribution. Our bounds are near-optimal in the case when the covariance is known to be the identity, and conjectured to be near-optimal in the general case. From a technical standpoint, we provide analytic tools for arguing the existence of global “locally small” covers from local covers of the space. These are exploited using modifications of recent techniques for differentially private hypothesis selection. Our techniques may prove useful for privately learning other distribution classes which do not possess a finite cover.

Cite this Paper

BibTeX
@InProceedings{pmlr-v132-aden-ali21a, title = {On the Sample Complexity of Privately Learning Unbounded High-Dimensional Gaussians}, author = {Aden-Ali, Ishaq and Ashtiani, Hassan and Kamath, Gautam}, booktitle = {Proceedings of the 32nd International Conference on Algorithmic Learning Theory}, pages = {185--216}, year = {2021}, editor = {Feldman, Vitaly and Ligett, Katrina and Sabato, Sivan}, volume = {132}, series = {Proceedings of Machine Learning Research}, month = {16--19 Mar}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v132/aden-ali21a/aden-ali21a.pdf}, url = {https://proceedings.mlr.press/v132/aden-ali21a.html}, abstract = {We provide sample complexity upper bounds for agnostically learning multivariate Gaussians under the constraint of approximate differential privacy. These are the first finite sample upper bounds for general Gaussians which do not impose restrictions on the parameters of the distribution. Our bounds are near-optimal in the case when the covariance is known to be the identity, and conjectured to be near-optimal in the general case. From a technical standpoint, we provide analytic tools for arguing the existence of global “locally small” covers from local covers of the space. These are exploited using modifications of recent techniques for differentially private hypothesis selection. Our techniques may prove useful for privately learning other distribution classes which do not possess a finite cover.} }
Endnote
%0 Conference Paper %T On the Sample Complexity of Privately Learning Unbounded High-Dimensional Gaussians %A Ishaq Aden-Ali %A Hassan Ashtiani %A Gautam Kamath %B Proceedings of the 32nd International Conference on Algorithmic Learning Theory %C Proceedings of Machine Learning Research %D 2021 %E Vitaly Feldman %E Katrina Ligett %E Sivan Sabato %F pmlr-v132-aden-ali21a %I PMLR %P 185--216 %U https://proceedings.mlr.press/v132/aden-ali21a.html %V 132 %X We provide sample complexity upper bounds for agnostically learning multivariate Gaussians under the constraint of approximate differential privacy. These are the first finite sample upper bounds for general Gaussians which do not impose restrictions on the parameters of the distribution. Our bounds are near-optimal in the case when the covariance is known to be the identity, and conjectured to be near-optimal in the general case. From a technical standpoint, we provide analytic tools for arguing the existence of global “locally small” covers from local covers of the space. These are exploited using modifications of recent techniques for differentially private hypothesis selection. Our techniques may prove useful for privately learning other distribution classes which do not possess a finite cover.
APA
Aden-Ali, I., Ashtiani, H. & Kamath, G.. (2021). On the Sample Complexity of Privately Learning Unbounded High-Dimensional Gaussians. Proceedings of the 32nd International Conference on Algorithmic Learning Theory, in Proceedings of Machine Learning Research 132:185-216 Available from https://proceedings.mlr.press/v132/aden-ali21a.html.

Related Material