Learning in Integer Latent Variable Models with Nested Automatic Differentiation

Daniel Sheldon, Kevin Winner, Debora Sujono
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:4615-4623, 2018.

Abstract

We develop nested automatic differentiation (AD) algorithms for exact inference and learning in integer latent variable models. Recently, Winner, Sujono, and Sheldon showed how to reduce marginalization in a class of integer latent variable models to evaluating a probability generating function which contains many levels of nested high-order derivatives. We contribute faster and more stable AD algorithms for this challenging problem and a novel algorithm to compute exact gradients for learning. These contributions lead to significantly faster and more accurate learning algorithms, and are the first AD algorithms whose running time is polynomial in the number of levels of nesting.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-sheldon18a, title = {Learning in Integer Latent Variable Models with Nested Automatic Differentiation}, author = {Sheldon, Daniel and Winner, Kevin and Sujono, Debora}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {4615--4623}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/sheldon18a/sheldon18a.pdf}, url = {https://proceedings.mlr.press/v80/sheldon18a.html}, abstract = {We develop nested automatic differentiation (AD) algorithms for exact inference and learning in integer latent variable models. Recently, Winner, Sujono, and Sheldon showed how to reduce marginalization in a class of integer latent variable models to evaluating a probability generating function which contains many levels of nested high-order derivatives. We contribute faster and more stable AD algorithms for this challenging problem and a novel algorithm to compute exact gradients for learning. These contributions lead to significantly faster and more accurate learning algorithms, and are the first AD algorithms whose running time is polynomial in the number of levels of nesting.} }
Endnote
%0 Conference Paper %T Learning in Integer Latent Variable Models with Nested Automatic Differentiation %A Daniel Sheldon %A Kevin Winner %A Debora Sujono %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-sheldon18a %I PMLR %P 4615--4623 %U https://proceedings.mlr.press/v80/sheldon18a.html %V 80 %X We develop nested automatic differentiation (AD) algorithms for exact inference and learning in integer latent variable models. Recently, Winner, Sujono, and Sheldon showed how to reduce marginalization in a class of integer latent variable models to evaluating a probability generating function which contains many levels of nested high-order derivatives. We contribute faster and more stable AD algorithms for this challenging problem and a novel algorithm to compute exact gradients for learning. These contributions lead to significantly faster and more accurate learning algorithms, and are the first AD algorithms whose running time is polynomial in the number of levels of nesting.
APA
Sheldon, D., Winner, K. & Sujono, D.. (2018). Learning in Integer Latent Variable Models with Nested Automatic Differentiation. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:4615-4623 Available from https://proceedings.mlr.press/v80/sheldon18a.html.

Related Material