Super-Samples from Kernel Herding

Yutian Chen, Max Welling, Alex Smola
Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, PMLR R8:117-124, 2010.

Abstract

We extend the herding algorithm to continuous spaces by using the kernel trick. The resulting “kernel herding” algorithm is an infinite mem- ory deterministic process that learns to approx- imate a PDF with a collection of samples. We show that kernel herding decreases the error of expectations of functions in the Hilbert space at a rate O(1/T ) which is much faster than the usual O(1/ $\sqrt{}$ T) for iid random samples. We illustrate kernel herding by approximating Bayesian pre- dictive distributions.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR8-chen10a, title = {Super-Samples from Kernel Herding}, author = {Chen, Yutian and Welling, Max and Smola, Alex}, booktitle = {Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence}, pages = {117--124}, year = {2010}, editor = {Grünwald, Peter and Spirtes, Peter}, volume = {R8}, series = {Proceedings of Machine Learning Research}, month = {08--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r8/main/assets/chen10a/chen10a.pdf}, url = {https://proceedings.mlr.press/r8/chen10a.html}, abstract = {We extend the herding algorithm to continuous spaces by using the kernel trick. The resulting “kernel herding” algorithm is an infinite mem- ory deterministic process that learns to approx- imate a PDF with a collection of samples. We show that kernel herding decreases the error of expectations of functions in the Hilbert space at a rate O(1/T ) which is much faster than the usual O(1/ $\sqrt{}$ T) for iid random samples. We illustrate kernel herding by approximating Bayesian pre- dictive distributions.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Super-Samples from Kernel Herding %A Yutian Chen %A Max Welling %A Alex Smola %B Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2010 %E Peter Grünwald %E Peter Spirtes %F pmlr-vR8-chen10a %I PMLR %P 117--124 %U https://proceedings.mlr.press/r8/chen10a.html %V R8 %X We extend the herding algorithm to continuous spaces by using the kernel trick. The resulting “kernel herding” algorithm is an infinite mem- ory deterministic process that learns to approx- imate a PDF with a collection of samples. We show that kernel herding decreases the error of expectations of functions in the Hilbert space at a rate O(1/T ) which is much faster than the usual O(1/ $\sqrt{}$ T) for iid random samples. We illustrate kernel herding by approximating Bayesian pre- dictive distributions. %Z Reissued by PMLR on 04 October 2026.
APA
Chen, Y., Welling, M. & Smola, A.. (2010). Super-Samples from Kernel Herding. Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R8:117-124 Available from https://proceedings.mlr.press/r8/chen10a.html. Reissued by PMLR on 04 October 2026.

Related Material