Inter and Intra Topic Structure Learning with Word Embeddings

He Zhao, Lan Du, Wray Buntine, Mingyuan Zhou
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:5892-5901, 2018.

Abstract

One important task of topic modeling for text analysis is interpretability. By discovering structured topics one is able to yield improved interpretability as well as modeling accuracy. In this paper, we propose a novel topic model with a deep structure that explores both inter-topic and intra-topic structures informed by word embeddings. Specifically, our model discovers inter topic structures in the form of topic hierarchies and discovers intra topic structures in the form of sub-topics, each of which is informed by word embeddings and captures a fine-grained thematic aspect of a normal topic. Extensive experiments demonstrate that our model achieves the state-of-the-art performance in terms of perplexity, document classification, and topic quality. Moreover, with topic hierarchies and sub-topics, the topics discovered in our model are more interpretable, providing an illuminating means to understand text data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-zhao18a, title = {Inter and Intra Topic Structure Learning with Word Embeddings}, author = {Zhao, He and Du, Lan and Buntine, Wray and Zhou, Mingyuan}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {5892--5901}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/zhao18a/zhao18a.pdf}, url = {https://proceedings.mlr.press/v80/zhao18a.html}, abstract = {One important task of topic modeling for text analysis is interpretability. By discovering structured topics one is able to yield improved interpretability as well as modeling accuracy. In this paper, we propose a novel topic model with a deep structure that explores both inter-topic and intra-topic structures informed by word embeddings. Specifically, our model discovers inter topic structures in the form of topic hierarchies and discovers intra topic structures in the form of sub-topics, each of which is informed by word embeddings and captures a fine-grained thematic aspect of a normal topic. Extensive experiments demonstrate that our model achieves the state-of-the-art performance in terms of perplexity, document classification, and topic quality. Moreover, with topic hierarchies and sub-topics, the topics discovered in our model are more interpretable, providing an illuminating means to understand text data.} }
Endnote
%0 Conference Paper %T Inter and Intra Topic Structure Learning with Word Embeddings %A He Zhao %A Lan Du %A Wray Buntine %A Mingyuan Zhou %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-zhao18a %I PMLR %P 5892--5901 %U https://proceedings.mlr.press/v80/zhao18a.html %V 80 %X One important task of topic modeling for text analysis is interpretability. By discovering structured topics one is able to yield improved interpretability as well as modeling accuracy. In this paper, we propose a novel topic model with a deep structure that explores both inter-topic and intra-topic structures informed by word embeddings. Specifically, our model discovers inter topic structures in the form of topic hierarchies and discovers intra topic structures in the form of sub-topics, each of which is informed by word embeddings and captures a fine-grained thematic aspect of a normal topic. Extensive experiments demonstrate that our model achieves the state-of-the-art performance in terms of perplexity, document classification, and topic quality. Moreover, with topic hierarchies and sub-topics, the topics discovered in our model are more interpretable, providing an illuminating means to understand text data.
APA
Zhao, H., Du, L., Buntine, W. & Zhou, M.. (2018). Inter and Intra Topic Structure Learning with Word Embeddings. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:5892-5901 Available from https://proceedings.mlr.press/v80/zhao18a.html.

Related Material