Stein Point Markov Chain Monte Carlo

Wilson Ye Chen, Alessandro Barp, Francois-Xavier Briol, Jackson Gorham, Mark Girolami, Lester Mackey, Chris Oates
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:1011-1021, 2019.

Abstract

An important task in machine learning and statistics is the approximation of a probability measure by an empirical measure supported on a discrete point set. Stein Points are a class of algorithms for this task, which proceed by sequentially minimising a Stein discrepancy between the empirical measure and the target and, hence, require the solution of a non-convex optimisation problem to obtain each new point. This paper removes the need to solve this optimisation problem by, instead, selecting each new point based on a Markov chain sample path. This significantly reduces the computational cost of Stein Points and leads to a suite of algorithms that are straightforward to implement. The new algorithms are illustrated on a set of challenging Bayesian inference problems, and rigorous theoretical guarantees of consistency are established.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-chen19b, title = {Stein Point {M}arkov Chain {M}onte {C}arlo}, author = {Chen, Wilson Ye and Barp, Alessandro and Briol, Francois-Xavier and Gorham, Jackson and Girolami, Mark and Mackey, Lester and Oates, Chris}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {1011--1021}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/chen19b/chen19b.pdf}, url = {https://proceedings.mlr.press/v97/chen19b.html}, abstract = {An important task in machine learning and statistics is the approximation of a probability measure by an empirical measure supported on a discrete point set. Stein Points are a class of algorithms for this task, which proceed by sequentially minimising a Stein discrepancy between the empirical measure and the target and, hence, require the solution of a non-convex optimisation problem to obtain each new point. This paper removes the need to solve this optimisation problem by, instead, selecting each new point based on a Markov chain sample path. This significantly reduces the computational cost of Stein Points and leads to a suite of algorithms that are straightforward to implement. The new algorithms are illustrated on a set of challenging Bayesian inference problems, and rigorous theoretical guarantees of consistency are established.} }
Endnote
%0 Conference Paper %T Stein Point Markov Chain Monte Carlo %A Wilson Ye Chen %A Alessandro Barp %A Francois-Xavier Briol %A Jackson Gorham %A Mark Girolami %A Lester Mackey %A Chris Oates %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-chen19b %I PMLR %P 1011--1021 %U https://proceedings.mlr.press/v97/chen19b.html %V 97 %X An important task in machine learning and statistics is the approximation of a probability measure by an empirical measure supported on a discrete point set. Stein Points are a class of algorithms for this task, which proceed by sequentially minimising a Stein discrepancy between the empirical measure and the target and, hence, require the solution of a non-convex optimisation problem to obtain each new point. This paper removes the need to solve this optimisation problem by, instead, selecting each new point based on a Markov chain sample path. This significantly reduces the computational cost of Stein Points and leads to a suite of algorithms that are straightforward to implement. The new algorithms are illustrated on a set of challenging Bayesian inference problems, and rigorous theoretical guarantees of consistency are established.
APA
Chen, W.Y., Barp, A., Briol, F., Gorham, J., Girolami, M., Mackey, L. & Oates, C.. (2019). Stein Point Markov Chain Monte Carlo. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:1011-1021 Available from https://proceedings.mlr.press/v97/chen19b.html.

Related Material