Particle algorithms for maximum likelihood training of latent variable models

Juan Kuntz, Jen Ning Lim, Adam M. Johansen
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:5134-5180, 2023.

Abstract

Neal and Hinton (1998) recast maximum likelihood estimation of any given latent variable model as the minimization of a free energy functional F, and the EM algorithm as coordinate descent applied to F. Here, we explore alternative ways to optimize the functional. In particular, we identify various gradient flows associated with F and show that their limits coincide with F’s stationary points. By discretizing the flows, we obtain practical particle-based algorithms for maximum likelihood estimation in broad classes of latent variable models. The novel algorithms scale to high-dimensional settings and perform well in numerical experiments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-kuntz23a, title = {Particle algorithms for maximum likelihood training of latent variable models}, author = {Kuntz, Juan and Lim, Jen Ning and Johansen, Adam M.}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {5134--5180}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/kuntz23a/kuntz23a.pdf}, url = {https://proceedings.mlr.press/v206/kuntz23a.html}, abstract = {Neal and Hinton (1998) recast maximum likelihood estimation of any given latent variable model as the minimization of a free energy functional F, and the EM algorithm as coordinate descent applied to F. Here, we explore alternative ways to optimize the functional. In particular, we identify various gradient flows associated with F and show that their limits coincide with F’s stationary points. By discretizing the flows, we obtain practical particle-based algorithms for maximum likelihood estimation in broad classes of latent variable models. The novel algorithms scale to high-dimensional settings and perform well in numerical experiments.} }
Endnote
%0 Conference Paper %T Particle algorithms for maximum likelihood training of latent variable models %A Juan Kuntz %A Jen Ning Lim %A Adam M. Johansen %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-kuntz23a %I PMLR %P 5134--5180 %U https://proceedings.mlr.press/v206/kuntz23a.html %V 206 %X Neal and Hinton (1998) recast maximum likelihood estimation of any given latent variable model as the minimization of a free energy functional F, and the EM algorithm as coordinate descent applied to F. Here, we explore alternative ways to optimize the functional. In particular, we identify various gradient flows associated with F and show that their limits coincide with F’s stationary points. By discretizing the flows, we obtain practical particle-based algorithms for maximum likelihood estimation in broad classes of latent variable models. The novel algorithms scale to high-dimensional settings and perform well in numerical experiments.
APA
Kuntz, J., Lim, J.N. & Johansen, A.M.. (2023). Particle algorithms for maximum likelihood training of latent variable models. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:5134-5180 Available from https://proceedings.mlr.press/v206/kuntz23a.html.

Related Material