Scalable Deep Poisson Factor Analysis for Topic Modeling

Zhe Gan, Changyou Chen, Ricardo Henao, David Carlson, Lawrence Carin
Proceedings of the 32nd International Conference on Machine Learning, PMLR 37:1823-1832, 2015.

Abstract

A new framework for topic modeling is developed, based on deep graphical models, where interactions between topics are inferred through deep latent binary hierarchies. The proposed multi-layer model employs a deep sigmoid belief network or restricted Boltzmann machine, the bottom binary layer of which selects topics for use in a Poisson factor analysis model. Under this setting, topics live on the bottom layer of the model, while the deep specification serves as a flexible prior for revealing topic structure. Scalable inference algorithms are derived by applying Bayesian conditional density filtering algorithm, in addition to extending recently proposed work on stochastic gradient thermostats. Experimental results on several corpora show that the proposed approach readily handles very large collections of text documents, infers structured topic representations, and obtains superior test perplexities when compared with related models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v37-gan15, title = {Scalable Deep Poisson Factor Analysis for Topic Modeling}, author = {Gan, Zhe and Chen, Changyou and Henao, Ricardo and Carlson, David and Carin, Lawrence}, booktitle = {Proceedings of the 32nd International Conference on Machine Learning}, pages = {1823--1832}, year = {2015}, editor = {Bach, Francis and Blei, David}, volume = {37}, series = {Proceedings of Machine Learning Research}, address = {Lille, France}, month = {07--09 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v37/gan15.pdf}, url = {https://proceedings.mlr.press/v37/gan15.html}, abstract = {A new framework for topic modeling is developed, based on deep graphical models, where interactions between topics are inferred through deep latent binary hierarchies. The proposed multi-layer model employs a deep sigmoid belief network or restricted Boltzmann machine, the bottom binary layer of which selects topics for use in a Poisson factor analysis model. Under this setting, topics live on the bottom layer of the model, while the deep specification serves as a flexible prior for revealing topic structure. Scalable inference algorithms are derived by applying Bayesian conditional density filtering algorithm, in addition to extending recently proposed work on stochastic gradient thermostats. Experimental results on several corpora show that the proposed approach readily handles very large collections of text documents, infers structured topic representations, and obtains superior test perplexities when compared with related models.} }
Endnote
%0 Conference Paper %T Scalable Deep Poisson Factor Analysis for Topic Modeling %A Zhe Gan %A Changyou Chen %A Ricardo Henao %A David Carlson %A Lawrence Carin %B Proceedings of the 32nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2015 %E Francis Bach %E David Blei %F pmlr-v37-gan15 %I PMLR %P 1823--1832 %U https://proceedings.mlr.press/v37/gan15.html %V 37 %X A new framework for topic modeling is developed, based on deep graphical models, where interactions between topics are inferred through deep latent binary hierarchies. The proposed multi-layer model employs a deep sigmoid belief network or restricted Boltzmann machine, the bottom binary layer of which selects topics for use in a Poisson factor analysis model. Under this setting, topics live on the bottom layer of the model, while the deep specification serves as a flexible prior for revealing topic structure. Scalable inference algorithms are derived by applying Bayesian conditional density filtering algorithm, in addition to extending recently proposed work on stochastic gradient thermostats. Experimental results on several corpora show that the proposed approach readily handles very large collections of text documents, infers structured topic representations, and obtains superior test perplexities when compared with related models.
RIS
TY - CPAPER TI - Scalable Deep Poisson Factor Analysis for Topic Modeling AU - Zhe Gan AU - Changyou Chen AU - Ricardo Henao AU - David Carlson AU - Lawrence Carin BT - Proceedings of the 32nd International Conference on Machine Learning DA - 2015/06/01 ED - Francis Bach ED - David Blei ID - pmlr-v37-gan15 PB - PMLR DP - Proceedings of Machine Learning Research VL - 37 SP - 1823 EP - 1832 L1 - http://proceedings.mlr.press/v37/gan15.pdf UR - https://proceedings.mlr.press/v37/gan15.html AB - A new framework for topic modeling is developed, based on deep graphical models, where interactions between topics are inferred through deep latent binary hierarchies. The proposed multi-layer model employs a deep sigmoid belief network or restricted Boltzmann machine, the bottom binary layer of which selects topics for use in a Poisson factor analysis model. Under this setting, topics live on the bottom layer of the model, while the deep specification serves as a flexible prior for revealing topic structure. Scalable inference algorithms are derived by applying Bayesian conditional density filtering algorithm, in addition to extending recently proposed work on stochastic gradient thermostats. Experimental results on several corpora show that the proposed approach readily handles very large collections of text documents, infers structured topic representations, and obtains superior test perplexities when compared with related models. ER -
APA
Gan, Z., Chen, C., Henao, R., Carlson, D. & Carin, L.. (2015). Scalable Deep Poisson Factor Analysis for Topic Modeling. Proceedings of the 32nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 37:1823-1832 Available from https://proceedings.mlr.press/v37/gan15.html.

Related Material