Exact Inference for Integer Latent-Variable Models

Kevin Winner, Debora Sujono, Dan Sheldon
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:3761-3770, 2017.

Abstract

Graphical models with latent count variables arise in a number of areas. However, standard inference algorithms do not apply to these models due to the infinite support of the latent variables. Winner and Sheldon (2016) recently developed a new technique using probability generating functions (PGFs) to perform efficient, exact inference for certain Poisson latent variable models. However, the method relies on symbolic manipulation of PGFs, and it is unclear whether this can be extended to more general models. In this paper we introduce a new approach for inference with PGFs: instead of manipulating PGFs symbolically, we adapt techniques from the autodiff literature to compute the higher-order derivatives necessary for inference. This substantially generalizes the class of models for which efficient, exact inference algorithms are available. Specifically, our results apply to a class of models that includes branching processes, which are widely used in applied mathematics and population ecology, and autoregressive models for integer data. Experiments show that our techniques are more scalable than existing approximate methods and enable new applications.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-winner17a, title = {Exact Inference for Integer Latent-Variable Models}, author = {Kevin Winner and Debora Sujono and Dan Sheldon}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {3761--3770}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/winner17a/winner17a.pdf}, url = {https://proceedings.mlr.press/v70/winner17a.html}, abstract = {Graphical models with latent count variables arise in a number of areas. However, standard inference algorithms do not apply to these models due to the infinite support of the latent variables. Winner and Sheldon (2016) recently developed a new technique using probability generating functions (PGFs) to perform efficient, exact inference for certain Poisson latent variable models. However, the method relies on symbolic manipulation of PGFs, and it is unclear whether this can be extended to more general models. In this paper we introduce a new approach for inference with PGFs: instead of manipulating PGFs symbolically, we adapt techniques from the autodiff literature to compute the higher-order derivatives necessary for inference. This substantially generalizes the class of models for which efficient, exact inference algorithms are available. Specifically, our results apply to a class of models that includes branching processes, which are widely used in applied mathematics and population ecology, and autoregressive models for integer data. Experiments show that our techniques are more scalable than existing approximate methods and enable new applications.} }
Endnote
%0 Conference Paper %T Exact Inference for Integer Latent-Variable Models %A Kevin Winner %A Debora Sujono %A Dan Sheldon %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-winner17a %I PMLR %P 3761--3770 %U https://proceedings.mlr.press/v70/winner17a.html %V 70 %X Graphical models with latent count variables arise in a number of areas. However, standard inference algorithms do not apply to these models due to the infinite support of the latent variables. Winner and Sheldon (2016) recently developed a new technique using probability generating functions (PGFs) to perform efficient, exact inference for certain Poisson latent variable models. However, the method relies on symbolic manipulation of PGFs, and it is unclear whether this can be extended to more general models. In this paper we introduce a new approach for inference with PGFs: instead of manipulating PGFs symbolically, we adapt techniques from the autodiff literature to compute the higher-order derivatives necessary for inference. This substantially generalizes the class of models for which efficient, exact inference algorithms are available. Specifically, our results apply to a class of models that includes branching processes, which are widely used in applied mathematics and population ecology, and autoregressive models for integer data. Experiments show that our techniques are more scalable than existing approximate methods and enable new applications.
APA
Winner, K., Sujono, D. & Sheldon, D.. (2017). Exact Inference for Integer Latent-Variable Models. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:3761-3770 Available from https://proceedings.mlr.press/v70/winner17a.html.

Related Material