Admixture of Poisson MRFs: A Topic Model with Word Dependencies

David Inouye, Pradeep Ravikumar, Inderjit Dhillon
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(1):683-691, 2014.

Abstract

This paper introduces a new topic model based on an admixture of Poisson Markov Random Fields (APM), which can model dependencies between words as opposed to previous independent topic models such as PLSA (Hofmann, 1999), LDA (Blei et al., 2003) or SAM (Reisinger et al., 2010). We propose a class of admixture models that generalizes previous topic models and show an equivalence between the conditional distribution of LDA and independent Poissons—suggesting that APM subsumes the modeling power of LDA. We present a tractable method for estimating the parameters of an APM based on the pseudo log-likelihood and demonstrate the benefits of APM over previous models by preliminary qualitative and quantitative experiments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-inouye14, title = {Admixture of Poisson MRFs: A Topic Model with Word Dependencies}, author = {Inouye, David and Ravikumar, Pradeep and Dhillon, Inderjit}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {683--691}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {1}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/inouye14.pdf}, url = {https://proceedings.mlr.press/v32/inouye14.html}, abstract = {This paper introduces a new topic model based on an admixture of Poisson Markov Random Fields (APM), which can model dependencies between words as opposed to previous independent topic models such as PLSA (Hofmann, 1999), LDA (Blei et al., 2003) or SAM (Reisinger et al., 2010). We propose a class of admixture models that generalizes previous topic models and show an equivalence between the conditional distribution of LDA and independent Poissons—suggesting that APM subsumes the modeling power of LDA. We present a tractable method for estimating the parameters of an APM based on the pseudo log-likelihood and demonstrate the benefits of APM over previous models by preliminary qualitative and quantitative experiments.} }
Endnote
%0 Conference Paper %T Admixture of Poisson MRFs: A Topic Model with Word Dependencies %A David Inouye %A Pradeep Ravikumar %A Inderjit Dhillon %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-inouye14 %I PMLR %P 683--691 %U https://proceedings.mlr.press/v32/inouye14.html %V 32 %N 1 %X This paper introduces a new topic model based on an admixture of Poisson Markov Random Fields (APM), which can model dependencies between words as opposed to previous independent topic models such as PLSA (Hofmann, 1999), LDA (Blei et al., 2003) or SAM (Reisinger et al., 2010). We propose a class of admixture models that generalizes previous topic models and show an equivalence between the conditional distribution of LDA and independent Poissons—suggesting that APM subsumes the modeling power of LDA. We present a tractable method for estimating the parameters of an APM based on the pseudo log-likelihood and demonstrate the benefits of APM over previous models by preliminary qualitative and quantitative experiments.
RIS
TY - CPAPER TI - Admixture of Poisson MRFs: A Topic Model with Word Dependencies AU - David Inouye AU - Pradeep Ravikumar AU - Inderjit Dhillon BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/01/27 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-inouye14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 1 SP - 683 EP - 691 L1 - http://proceedings.mlr.press/v32/inouye14.pdf UR - https://proceedings.mlr.press/v32/inouye14.html AB - This paper introduces a new topic model based on an admixture of Poisson Markov Random Fields (APM), which can model dependencies between words as opposed to previous independent topic models such as PLSA (Hofmann, 1999), LDA (Blei et al., 2003) or SAM (Reisinger et al., 2010). We propose a class of admixture models that generalizes previous topic models and show an equivalence between the conditional distribution of LDA and independent Poissons—suggesting that APM subsumes the modeling power of LDA. We present a tractable method for estimating the parameters of an APM based on the pseudo log-likelihood and demonstrate the benefits of APM over previous models by preliminary qualitative and quantitative experiments. ER -
APA
Inouye, D., Ravikumar, P. & Dhillon, I.. (2014). Admixture of Poisson MRFs: A Topic Model with Word Dependencies. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(1):683-691 Available from https://proceedings.mlr.press/v32/inouye14.html.

Related Material