Optimal Quantisation of Probability Measures Using Maximum Mean Discrepancy

Onur Teymur, Jackson Gorham, Marina Riabiz, Chris Oates
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:1027-1035, 2021.

Abstract

Several researchers have proposed minimisation of maximum mean discrepancy (MMD) as a method to quantise probability measures, i.e., to approximate a distribution by a representative point set. We consider sequential algorithms that greedily minimise MMD over a discrete candidate set. We propose a novel non-myopic algorithm and, in order to both improve statistical efficiency and reduce computational cost, we investigate a variant that applies this technique to a mini-batch of the candidate set at each iteration. When the candidate points are sampled from the target, the consistency of these new algorithms—and their mini-batch variants—is established. We demonstrate the algorithms on a range of important computational problems, including optimisation of nodes in Bayesian cubature and the thinning of Markov chain output.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-teymur21a, title = { Optimal Quantisation of Probability Measures Using Maximum Mean Discrepancy }, author = {Teymur, Onur and Gorham, Jackson and Riabiz, Marina and Oates, Chris}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {1027--1035}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/teymur21a/teymur21a.pdf}, url = {https://proceedings.mlr.press/v130/teymur21a.html}, abstract = { Several researchers have proposed minimisation of maximum mean discrepancy (MMD) as a method to quantise probability measures, i.e., to approximate a distribution by a representative point set. We consider sequential algorithms that greedily minimise MMD over a discrete candidate set. We propose a novel non-myopic algorithm and, in order to both improve statistical efficiency and reduce computational cost, we investigate a variant that applies this technique to a mini-batch of the candidate set at each iteration. When the candidate points are sampled from the target, the consistency of these new algorithms—and their mini-batch variants—is established. We demonstrate the algorithms on a range of important computational problems, including optimisation of nodes in Bayesian cubature and the thinning of Markov chain output. } }
Endnote
%0 Conference Paper %T Optimal Quantisation of Probability Measures Using Maximum Mean Discrepancy %A Onur Teymur %A Jackson Gorham %A Marina Riabiz %A Chris Oates %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-teymur21a %I PMLR %P 1027--1035 %U https://proceedings.mlr.press/v130/teymur21a.html %V 130 %X Several researchers have proposed minimisation of maximum mean discrepancy (MMD) as a method to quantise probability measures, i.e., to approximate a distribution by a representative point set. We consider sequential algorithms that greedily minimise MMD over a discrete candidate set. We propose a novel non-myopic algorithm and, in order to both improve statistical efficiency and reduce computational cost, we investigate a variant that applies this technique to a mini-batch of the candidate set at each iteration. When the candidate points are sampled from the target, the consistency of these new algorithms—and their mini-batch variants—is established. We demonstrate the algorithms on a range of important computational problems, including optimisation of nodes in Bayesian cubature and the thinning of Markov chain output.
APA
Teymur, O., Gorham, J., Riabiz, M. & Oates, C.. (2021). Optimal Quantisation of Probability Measures Using Maximum Mean Discrepancy . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:1027-1035 Available from https://proceedings.mlr.press/v130/teymur21a.html.

Related Material