Differentially Private Distributed Bayesian Linear Regression with MCMC

Baris Alparslan, Sinan Yıldırım, Ilker Birbil
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:627-641, 2023.

Abstract

We propose a novel Bayesian inference framework for distributed differentially private linear regression. We consider a distributed setting where multiple parties hold parts of the data and share certain summary statistics of their portions in privacy-preserving noise. We develop a novel generative statistical model for privately shared statistics, which exploits a useful distributional relation between the summary statistics of linear regression. We propose Bayesian estimation of the regression coefficients, mainly using Markov chain Monte Carlo algorithms, while we also provide a fast version that performs approximate Bayesian estimation in one iteration. The proposed methods have computational advantages over their competitors. We provide numerical results on both real and simulated data, which demonstrate that the proposed algorithms provide well-rounded estimation and prediction.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-alparslan23a, title = {Differentially Private Distributed {B}ayesian Linear Regression with {MCMC}}, author = {Alparslan, Baris and Y{\i}ld{\i}r{\i}m, Sinan and Birbil, Ilker}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {627--641}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/alparslan23a/alparslan23a.pdf}, url = {https://proceedings.mlr.press/v202/alparslan23a.html}, abstract = {We propose a novel Bayesian inference framework for distributed differentially private linear regression. We consider a distributed setting where multiple parties hold parts of the data and share certain summary statistics of their portions in privacy-preserving noise. We develop a novel generative statistical model for privately shared statistics, which exploits a useful distributional relation between the summary statistics of linear regression. We propose Bayesian estimation of the regression coefficients, mainly using Markov chain Monte Carlo algorithms, while we also provide a fast version that performs approximate Bayesian estimation in one iteration. The proposed methods have computational advantages over their competitors. We provide numerical results on both real and simulated data, which demonstrate that the proposed algorithms provide well-rounded estimation and prediction.} }
Endnote
%0 Conference Paper %T Differentially Private Distributed Bayesian Linear Regression with MCMC %A Baris Alparslan %A Sinan Yıldırım %A Ilker Birbil %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-alparslan23a %I PMLR %P 627--641 %U https://proceedings.mlr.press/v202/alparslan23a.html %V 202 %X We propose a novel Bayesian inference framework for distributed differentially private linear regression. We consider a distributed setting where multiple parties hold parts of the data and share certain summary statistics of their portions in privacy-preserving noise. We develop a novel generative statistical model for privately shared statistics, which exploits a useful distributional relation between the summary statistics of linear regression. We propose Bayesian estimation of the regression coefficients, mainly using Markov chain Monte Carlo algorithms, while we also provide a fast version that performs approximate Bayesian estimation in one iteration. The proposed methods have computational advantages over their competitors. We provide numerical results on both real and simulated data, which demonstrate that the proposed algorithms provide well-rounded estimation and prediction.
APA
Alparslan, B., Yıldırım, S. & Birbil, I.. (2023). Differentially Private Distributed Bayesian Linear Regression with MCMC. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:627-641 Available from https://proceedings.mlr.press/v202/alparslan23a.html.

Related Material