Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective

Jacky Zhang, Rajiv Khanna, Anastasios Kyrillidis, Sanmi Koyejo
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:2782-2790, 2021.

Abstract

Bayesian coresets have emerged as a promising approach for implementing scalable Bayesian inference. The Bayesian coreset problem involves selecting a (weighted) subset of the data samples, such that the posterior inference using the selected subset closely approximates the posterior inference using the full dataset. This manuscript revisits Bayesian coresets through the lens of sparsity constrained optimization. Leveraging recent advances in accelerated optimization methods, we propose and analyze a novel algorithm for coreset selection. We provide explicit convergence rate guarantees and present an empirical evaluation on a variety of benchmark datasets to highlight our proposed algorithm’s superior performance compared to state-of-the-art on speed and accuracy.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-zhang21g, title = { Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective }, author = {Zhang, Jacky and Khanna, Rajiv and Kyrillidis, Anastasios and Koyejo, Sanmi}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {2782--2790}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/zhang21g/zhang21g.pdf}, url = {https://proceedings.mlr.press/v130/zhang21g.html}, abstract = { Bayesian coresets have emerged as a promising approach for implementing scalable Bayesian inference. The Bayesian coreset problem involves selecting a (weighted) subset of the data samples, such that the posterior inference using the selected subset closely approximates the posterior inference using the full dataset. This manuscript revisits Bayesian coresets through the lens of sparsity constrained optimization. Leveraging recent advances in accelerated optimization methods, we propose and analyze a novel algorithm for coreset selection. We provide explicit convergence rate guarantees and present an empirical evaluation on a variety of benchmark datasets to highlight our proposed algorithm’s superior performance compared to state-of-the-art on speed and accuracy. } }
Endnote
%0 Conference Paper %T Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective %A Jacky Zhang %A Rajiv Khanna %A Anastasios Kyrillidis %A Sanmi Koyejo %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-zhang21g %I PMLR %P 2782--2790 %U https://proceedings.mlr.press/v130/zhang21g.html %V 130 %X Bayesian coresets have emerged as a promising approach for implementing scalable Bayesian inference. The Bayesian coreset problem involves selecting a (weighted) subset of the data samples, such that the posterior inference using the selected subset closely approximates the posterior inference using the full dataset. This manuscript revisits Bayesian coresets through the lens of sparsity constrained optimization. Leveraging recent advances in accelerated optimization methods, we propose and analyze a novel algorithm for coreset selection. We provide explicit convergence rate guarantees and present an empirical evaluation on a variety of benchmark datasets to highlight our proposed algorithm’s superior performance compared to state-of-the-art on speed and accuracy.
APA
Zhang, J., Khanna, R., Kyrillidis, A. & Koyejo, S.. (2021). Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:2782-2790 Available from https://proceedings.mlr.press/v130/zhang21g.html.

Related Material