Combinatorial Topic Models using Small-Variance Asymptotics


Ke Jiang, Suvrit Sra, Brian Kulis ;
Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR 54:421-429, 2017.


Modern topic models typically have a probabilistic formulation, and derive their inference algorithms based on Latent Dirichlet Allocation (LDA) and its variants. In contrast, we approach topic modeling via combinatorial optimization, and take a small-variance limit of LDA to derive a new objective function. We minimize this objective by using ideas from combinatorial optimization, obtaining a new, fast, and high-quality topic modeling algorithm. In particular, we show that our results are not only significantly better than traditional SVA algorithms, but also truly competitive with popular LDA-based approaches; we also discuss the (dis)similarities between our approach and its probabilistic counterparts.

Related Material