Herding Dynamic Weights for Partially Observed Random Field Models

Max Welling
Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, PMLR R7:599-606, 2009.

Abstract

Learning the parameters of a (potentially partially observable) random field model is intractable in general. Instead of focussing on a single optimal parameter value we propose to treat parameters as dynamical quantities. We introduce an algorithm to generate complex dynamics for parameters and (both visible and hidden) state vectors. We show that under certain conditions averages computed over trajectories of the proposed dynamical system converge to averages computed over the data. Our "herding dynamics" does not require expensive operations such as exponentiation and is fully deterministic.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR7-welling09a, title = {Herding Dynamic Weights for Partially Observed Random Field Models}, author = {Welling, Max}, booktitle = {Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence}, pages = {599--606}, year = {2009}, editor = {Bilmes, Jeff and Ng, Andrew Y.}, volume = {R7}, series = {Proceedings of Machine Learning Research}, month = {18--21 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r7/main/assets/welling09a/welling09a.pdf}, url = {https://proceedings.mlr.press/r7/welling09a.html}, abstract = {Learning the parameters of a (potentially partially observable) random field model is intractable in general. Instead of focussing on a single optimal parameter value we propose to treat parameters as dynamical quantities. We introduce an algorithm to generate complex dynamics for parameters and (both visible and hidden) state vectors. We show that under certain conditions averages computed over trajectories of the proposed dynamical system converge to averages computed over the data. Our "herding dynamics" does not require expensive operations such as exponentiation and is fully deterministic.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Herding Dynamic Weights for Partially Observed Random Field Models %A Max Welling %B Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2009 %E Jeff Bilmes %E Andrew Y. Ng %F pmlr-vR7-welling09a %I PMLR %P 599--606 %U https://proceedings.mlr.press/r7/welling09a.html %V R7 %X Learning the parameters of a (potentially partially observable) random field model is intractable in general. Instead of focussing on a single optimal parameter value we propose to treat parameters as dynamical quantities. We introduce an algorithm to generate complex dynamics for parameters and (both visible and hidden) state vectors. We show that under certain conditions averages computed over trajectories of the proposed dynamical system converge to averages computed over the data. Our "herding dynamics" does not require expensive operations such as exponentiation and is fully deterministic. %Z Reissued by PMLR on 04 October 2026.
APA
Welling, M.. (2009). Herding Dynamic Weights for Partially Observed Random Field Models. Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R7:599-606 Available from https://proceedings.mlr.press/r7/welling09a.html. Reissued by PMLR on 04 October 2026.

Related Material