Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain

Yu-Xiang Wang
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:92-102, 2018.

Abstract

We revisit the problem of linear regression under a differential privacy constraint. By consolidating existing pieces in the literature, we clarify the correct dependence of the feature, label and coefficient domains in the optimization error and estimation error, hence revealing the delicate price of differential privacy in statistical estimation and statistical learning. Moreover, we propose simple modifications of two existing DP algorithms: (a) posterior sampling, (b) sufficient statistics perturbation, and show that they can be upgraded into adaptive algorithms that are able to exploit data-dependent quantities and behave nearly optimally for every instance. Extensive experiments are conducted on both simulated data and real data, which conclude that both ADAOPS and ADASSP outperform the existing techniques on nearly all 36 data sets that we test on.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-wang18b, title = {Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain}, author = {Wang, Yu-Xiang}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {92--102}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/wang18b/wang18b.pdf}, url = {https://proceedings.mlr.press/r16/wang18b.html}, abstract = {We revisit the problem of linear regression under a differential privacy constraint. By consolidating existing pieces in the literature, we clarify the correct dependence of the feature, label and coefficient domains in the optimization error and estimation error, hence revealing the delicate price of differential privacy in statistical estimation and statistical learning. Moreover, we propose simple modifications of two existing DP algorithms: (a) posterior sampling, (b) sufficient statistics perturbation, and show that they can be upgraded into adaptive algorithms that are able to exploit data-dependent quantities and behave nearly optimally for every instance. Extensive experiments are conducted on both simulated data and real data, which conclude that both ADAOPS and ADASSP outperform the existing techniques on nearly all 36 data sets that we test on.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain %A Yu-Xiang Wang %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-wang18b %I PMLR %P 92--102 %U https://proceedings.mlr.press/r16/wang18b.html %V R16 %X We revisit the problem of linear regression under a differential privacy constraint. By consolidating existing pieces in the literature, we clarify the correct dependence of the feature, label and coefficient domains in the optimization error and estimation error, hence revealing the delicate price of differential privacy in statistical estimation and statistical learning. Moreover, we propose simple modifications of two existing DP algorithms: (a) posterior sampling, (b) sufficient statistics perturbation, and show that they can be upgraded into adaptive algorithms that are able to exploit data-dependent quantities and behave nearly optimally for every instance. Extensive experiments are conducted on both simulated data and real data, which conclude that both ADAOPS and ADASSP outperform the existing techniques on nearly all 36 data sets that we test on. %Z Reissued by PMLR on 04 October 2026.
APA
Wang, Y.. (2018). Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:92-102 Available from https://proceedings.mlr.press/r16/wang18b.html. Reissued by PMLR on 04 October 2026.

Related Material