The Discrete Infinite Logistic Normal Distribution for Mixed-Membership Modeling

John Paisley, Chong Wang, David Blei
Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, PMLR 15:74-82, 2011.

Abstract

We present the discrete infinite logistic normal distribution (DILN, “Dylan”), a Bayesian nonparametric prior for mixed membership models. DILN is a generalization of the hierarchical Dirichlet process (HDP) that models correlation structure between the weights of the atoms at the group level. We derive a representation of DILN as a normalized collection of gamma-distributed random variables, and study its statistical properties. We consider applications to topic modeling and derive a variational Bayes algorithm for approximate posterior inference. We study the empirical performance of the DILN topic model on four corpora, comparing performance with the HDP and the correlated topic model.

Cite this Paper

BibTeX
@InProceedings{pmlr-v15-paisley11a, title = {The Discrete Infinite Logistic Normal Distribution for Mixed-Membership Modeling}, author = {Paisley, John and Wang, Chong and Blei, David}, booktitle = {Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics}, pages = {74--82}, year = {2011}, editor = {Gordon, Geoffrey and Dunson, David and Dudík, Miroslav}, volume = {15}, series = {Proceedings of Machine Learning Research}, address = {Fort Lauderdale, FL, USA}, month = {11--13 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v15/paisley11a/paisley11a.pdf}, url = {https://proceedings.mlr.press/v15/paisley11a.html}, abstract = {We present the discrete infinite logistic normal distribution (DILN, “Dylan”), a Bayesian nonparametric prior for mixed membership models. DILN is a generalization of the hierarchical Dirichlet process (HDP) that models correlation structure between the weights of the atoms at the group level. We derive a representation of DILN as a normalized collection of gamma-distributed random variables, and study its statistical properties. We consider applications to topic modeling and derive a variational Bayes algorithm for approximate posterior inference. We study the empirical performance of the DILN topic model on four corpora, comparing performance with the HDP and the correlated topic model. } }
Endnote
%0 Conference Paper %T The Discrete Infinite Logistic Normal Distribution for Mixed-Membership Modeling %A John Paisley %A Chong Wang %A David Blei %B Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2011 %E Geoffrey Gordon %E David Dunson %E Miroslav Dudík %F pmlr-v15-paisley11a %I PMLR %P 74--82 %U https://proceedings.mlr.press/v15/paisley11a.html %V 15 %X We present the discrete infinite logistic normal distribution (DILN, “Dylan”), a Bayesian nonparametric prior for mixed membership models. DILN is a generalization of the hierarchical Dirichlet process (HDP) that models correlation structure between the weights of the atoms at the group level. We derive a representation of DILN as a normalized collection of gamma-distributed random variables, and study its statistical properties. We consider applications to topic modeling and derive a variational Bayes algorithm for approximate posterior inference. We study the empirical performance of the DILN topic model on four corpora, comparing performance with the HDP and the correlated topic model.
RIS
TY - CPAPER TI - The Discrete Infinite Logistic Normal Distribution for Mixed-Membership Modeling AU - John Paisley AU - Chong Wang AU - David Blei BT - Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics DA - 2011/06/14 ED - Geoffrey Gordon ED - David Dunson ED - Miroslav Dudík ID - pmlr-v15-paisley11a PB - PMLR DP - Proceedings of Machine Learning Research VL - 15 SP - 74 EP - 82 L1 - http://proceedings.mlr.press/v15/paisley11a/paisley11a.pdf UR - https://proceedings.mlr.press/v15/paisley11a.html AB - We present the discrete infinite logistic normal distribution (DILN, “Dylan”), a Bayesian nonparametric prior for mixed membership models. DILN is a generalization of the hierarchical Dirichlet process (HDP) that models correlation structure between the weights of the atoms at the group level. We derive a representation of DILN as a normalized collection of gamma-distributed random variables, and study its statistical properties. We consider applications to topic modeling and derive a variational Bayes algorithm for approximate posterior inference. We study the empirical performance of the DILN topic model on four corpora, comparing performance with the HDP and the correlated topic model. ER -
APA
Paisley, J., Wang, C. & Blei, D.. (2011). The Discrete Infinite Logistic Normal Distribution for Mixed-Membership Modeling. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 15:74-82 Available from https://proceedings.mlr.press/v15/paisley11a.html.

Related Material