Stein Points

Wilson Ye Chen, Lester Mackey, Jackson Gorham, Francois-Xavier Briol, Chris Oates
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:844-853, 2018.

Abstract

An important task in computational statistics and machine learning is to approximate a posterior distribution $p(x)$ with an empirical measure supported on a set of representative points $\{x_i\}_{i=1}^n$. This paper focuses on methods where the selection of points is essentially deterministic, with an emphasis on achieving accurate approximation when $n$ is small. To this end, we present Stein Points. The idea is to exploit either a greedy or a conditional gradient method to iteratively minimise a kernel Stein discrepancy between the empirical measure and $p(x)$. Our empirical results demonstrate that Stein Points enable accurate approximation of the posterior at modest computational cost. In addition, theoretical results are provided to establish convergence of the method.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-chen18f, title = {Stein Points}, author = {Chen, Wilson Ye and Mackey, Lester and Gorham, Jackson and Briol, Francois-Xavier and Oates, Chris}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {844--853}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/chen18f/chen18f.pdf}, url = {https://proceedings.mlr.press/v80/chen18f.html}, abstract = {An important task in computational statistics and machine learning is to approximate a posterior distribution $p(x)$ with an empirical measure supported on a set of representative points $\{x_i\}_{i=1}^n$. This paper focuses on methods where the selection of points is essentially deterministic, with an emphasis on achieving accurate approximation when $n$ is small. To this end, we present Stein Points. The idea is to exploit either a greedy or a conditional gradient method to iteratively minimise a kernel Stein discrepancy between the empirical measure and $p(x)$. Our empirical results demonstrate that Stein Points enable accurate approximation of the posterior at modest computational cost. In addition, theoretical results are provided to establish convergence of the method.} }
Endnote
%0 Conference Paper %T Stein Points %A Wilson Ye Chen %A Lester Mackey %A Jackson Gorham %A Francois-Xavier Briol %A Chris Oates %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-chen18f %I PMLR %P 844--853 %U https://proceedings.mlr.press/v80/chen18f.html %V 80 %X An important task in computational statistics and machine learning is to approximate a posterior distribution $p(x)$ with an empirical measure supported on a set of representative points $\{x_i\}_{i=1}^n$. This paper focuses on methods where the selection of points is essentially deterministic, with an emphasis on achieving accurate approximation when $n$ is small. To this end, we present Stein Points. The idea is to exploit either a greedy or a conditional gradient method to iteratively minimise a kernel Stein discrepancy between the empirical measure and $p(x)$. Our empirical results demonstrate that Stein Points enable accurate approximation of the posterior at modest computational cost. In addition, theoretical results are provided to establish convergence of the method.
APA
Chen, W.Y., Mackey, L., Gorham, J., Briol, F. & Oates, C.. (2018). Stein Points. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:844-853 Available from https://proceedings.mlr.press/v80/chen18f.html.

Related Material