Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates

Louis Sharrock, Christopher Nemeth
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:30850-30882, 2023.

Abstract

In recent years, particle-based variational inference (ParVI) methods such as Stein variational gradient descent (SVGD) have grown in popularity as scalable methods for Bayesian inference. Unfortunately, the properties of such methods invariably depend on hyperparameters such as the learning rate, which must be carefully tuned by the practitioner in order to ensure convergence to the target measure at a suitable rate. In this paper, we introduce a suite of new particle-based methods for scalable Bayesian inference based on coin betting, which are entirely learning-rate free. We illustrate the performance of our approach on a range of numerical examples, including several high-dimensional models and datasets, demonstrating comparable performance to other ParVI algorithms with no need to tune a learning rate.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-sharrock23a, title = {Coin Sampling: Gradient-Based {B}ayesian Inference without Learning Rates}, author = {Sharrock, Louis and Nemeth, Christopher}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {30850--30882}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/sharrock23a/sharrock23a.pdf}, url = {https://proceedings.mlr.press/v202/sharrock23a.html}, abstract = {In recent years, particle-based variational inference (ParVI) methods such as Stein variational gradient descent (SVGD) have grown in popularity as scalable methods for Bayesian inference. Unfortunately, the properties of such methods invariably depend on hyperparameters such as the learning rate, which must be carefully tuned by the practitioner in order to ensure convergence to the target measure at a suitable rate. In this paper, we introduce a suite of new particle-based methods for scalable Bayesian inference based on coin betting, which are entirely learning-rate free. We illustrate the performance of our approach on a range of numerical examples, including several high-dimensional models and datasets, demonstrating comparable performance to other ParVI algorithms with no need to tune a learning rate.} }
Endnote
%0 Conference Paper %T Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates %A Louis Sharrock %A Christopher Nemeth %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-sharrock23a %I PMLR %P 30850--30882 %U https://proceedings.mlr.press/v202/sharrock23a.html %V 202 %X In recent years, particle-based variational inference (ParVI) methods such as Stein variational gradient descent (SVGD) have grown in popularity as scalable methods for Bayesian inference. Unfortunately, the properties of such methods invariably depend on hyperparameters such as the learning rate, which must be carefully tuned by the practitioner in order to ensure convergence to the target measure at a suitable rate. In this paper, we introduce a suite of new particle-based methods for scalable Bayesian inference based on coin betting, which are entirely learning-rate free. We illustrate the performance of our approach on a range of numerical examples, including several high-dimensional models and datasets, demonstrating comparable performance to other ParVI algorithms with no need to tune a learning rate.
APA
Sharrock, L. & Nemeth, C.. (2023). Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:30850-30882 Available from https://proceedings.mlr.press/v202/sharrock23a.html.

Related Material