Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap

Edwin Fong, Simon Lyddon, Chris Holmes
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:1952-1962, 2019.

Abstract

Increasingly complex datasets pose a number of challenges for Bayesian inference. Conventional posterior sampling based on Markov chain Monte Carlo can be too computationally intensive, is serial in nature and mixes poorly between posterior modes. Furthermore, all models are misspecified, which brings into question the validity of the conventional Bayesian update. We present a scalable Bayesian nonparametric learning routine that enables posterior sampling through the optimization of suitably randomized objective functions. A Dirichlet process prior on the unknown data distribution accounts for model misspecification, and admits an embarrassingly parallel posterior bootstrap algorithm that generates independent and exact samples from the nonparametric posterior distribution. Our method is particularly adept at sampling from multimodal posterior distributions via a random restart mechanism, and we demonstrate this on Gaussian mixture model and sparse logistic regression examples.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-fong19a, title = {Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap}, author = {Fong, Edwin and Lyddon, Simon and Holmes, Chris}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {1952--1962}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/fong19a/fong19a.pdf}, url = {https://proceedings.mlr.press/v97/fong19a.html}, abstract = {Increasingly complex datasets pose a number of challenges for Bayesian inference. Conventional posterior sampling based on Markov chain Monte Carlo can be too computationally intensive, is serial in nature and mixes poorly between posterior modes. Furthermore, all models are misspecified, which brings into question the validity of the conventional Bayesian update. We present a scalable Bayesian nonparametric learning routine that enables posterior sampling through the optimization of suitably randomized objective functions. A Dirichlet process prior on the unknown data distribution accounts for model misspecification, and admits an embarrassingly parallel posterior bootstrap algorithm that generates independent and exact samples from the nonparametric posterior distribution. Our method is particularly adept at sampling from multimodal posterior distributions via a random restart mechanism, and we demonstrate this on Gaussian mixture model and sparse logistic regression examples.} }
Endnote
%0 Conference Paper %T Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap %A Edwin Fong %A Simon Lyddon %A Chris Holmes %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-fong19a %I PMLR %P 1952--1962 %U https://proceedings.mlr.press/v97/fong19a.html %V 97 %X Increasingly complex datasets pose a number of challenges for Bayesian inference. Conventional posterior sampling based on Markov chain Monte Carlo can be too computationally intensive, is serial in nature and mixes poorly between posterior modes. Furthermore, all models are misspecified, which brings into question the validity of the conventional Bayesian update. We present a scalable Bayesian nonparametric learning routine that enables posterior sampling through the optimization of suitably randomized objective functions. A Dirichlet process prior on the unknown data distribution accounts for model misspecification, and admits an embarrassingly parallel posterior bootstrap algorithm that generates independent and exact samples from the nonparametric posterior distribution. Our method is particularly adept at sampling from multimodal posterior distributions via a random restart mechanism, and we demonstrate this on Gaussian mixture model and sparse logistic regression examples.
APA
Fong, E., Lyddon, S. & Holmes, C.. (2019). Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:1952-1962 Available from https://proceedings.mlr.press/v97/fong19a.html.

Related Material