The Discrete Infinite Logistic Normal Distribution for Mixed-Membership Modeling


John Paisley, Chong Wang, David Blei ;
Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, PMLR 15:74-82, 2011.


We present the discrete infinite logistic normal distribution (DILN, “"Dylan""), a Bayesian nonparametric prior for mixed membership models. DILN is a generalization of the hierarchical Dirichlet process (HDP) that models correlation structure between the weights of the atoms at the group level. We derive a representation of DILN as a normalized collection of gamma-distributed random variables, and study its statistical properties. We consider applications to topic modeling and derive a variational Bayes algorithm for approximate posterior inference. We study the empirical performance of the DILN topic model on four corpora, comparing performance with the HDP and the correlated topic model.

Related Material