Optimally-Weighted Herding is Bayesian Quadrature

Ferenc Huszar, David Duvenaud
Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, PMLR R10:375-384, 2012.

Abstract

Herding and kernel herding are deterministic methods of choosing samples which summarise a probability distribution. A related task is choosing samples for estimating integrals using Bayesian quadrature. We show that the criterion minimised when selecting samples in kernel herding is equivalent to the posterior variance in Bayesian quadrature. We then show that sequential Bayesian quadrature can be viewed as a weighted version of kernel herding which achieves performance superior to any other weighted herding method. We demonstrate empirically a rate of convergence faster than O(1/N). Our results also imply an upper bound on the empirical error of the Bayesian quadrature estimate.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR10-huszar12a, title = {Optimally-Weighted Herding is {B}ayesian Quadrature}, author = {Huszar, Ferenc and Duvenaud, David}, booktitle = {Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence}, pages = {375--384}, year = {2012}, editor = {de Freitas, Nando and Murphy, Kevin}, volume = {R10}, series = {Proceedings of Machine Learning Research}, month = {14--18 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r10/main/assets/huszar12a/huszar12a.pdf}, url = {https://proceedings.mlr.press/r10/huszar12a.html}, abstract = {Herding and kernel herding are deterministic methods of choosing samples which summarise a probability distribution. A related task is choosing samples for estimating integrals using Bayesian quadrature. We show that the criterion minimised when selecting samples in kernel herding is equivalent to the posterior variance in Bayesian quadrature. We then show that sequential Bayesian quadrature can be viewed as a weighted version of kernel herding which achieves performance superior to any other weighted herding method. We demonstrate empirically a rate of convergence faster than O(1/N). Our results also imply an upper bound on the empirical error of the Bayesian quadrature estimate.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Optimally-Weighted Herding is Bayesian Quadrature %A Ferenc Huszar %A David Duvenaud %B Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2012 %E Nando de Freitas %E Kevin Murphy %F pmlr-vR10-huszar12a %I PMLR %P 375--384 %U https://proceedings.mlr.press/r10/huszar12a.html %V R10 %X Herding and kernel herding are deterministic methods of choosing samples which summarise a probability distribution. A related task is choosing samples for estimating integrals using Bayesian quadrature. We show that the criterion minimised when selecting samples in kernel herding is equivalent to the posterior variance in Bayesian quadrature. We then show that sequential Bayesian quadrature can be viewed as a weighted version of kernel herding which achieves performance superior to any other weighted herding method. We demonstrate empirically a rate of convergence faster than O(1/N). Our results also imply an upper bound on the empirical error of the Bayesian quadrature estimate. %Z Reissued by PMLR on 04 October 2026.
APA
Huszar, F. & Duvenaud, D.. (2012). Optimally-Weighted Herding is Bayesian Quadrature. Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R10:375-384 Available from https://proceedings.mlr.press/r10/huszar12a.html. Reissued by PMLR on 04 October 2026.

Related Material