Bayesian Deep Embedding Topic Meta-Learner

Zhibin Duan, Yishi Xu, Jianqiao Sun, Bo Chen, Wenchao Chen, Chaojie Wang, Mingyuan Zhou
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:5659-5670, 2022.

Abstract

Existing deep topic models are effective in capturing the latent semantic structures in textual data but usually rely on a plethora of documents. This is less than satisfactory in practical applications when only a limited amount of data is available. In this paper, we propose a novel framework that efficiently solves the problem of topic modeling under the small data regime. Specifically, the framework involves two innovations: a bi-level generative model that aims to exploit the task information to guide the document generation, and a topic meta-learner that strives to learn a group of global topic embeddings so that fast adaptation to the task-specific topic embeddings can be achieved with a few examples. We apply the proposed framework to a hierarchical embedded topic model and achieve better performance than various baseline models on diverse experiments, including few-shot topic discovery and few-shot document classification.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-duan22d, title = {{B}ayesian Deep Embedding Topic Meta-Learner}, author = {Duan, Zhibin and Xu, Yishi and Sun, Jianqiao and Chen, Bo and Chen, Wenchao and Wang, Chaojie and Zhou, Mingyuan}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {5659--5670}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/duan22d/duan22d.pdf}, url = {https://proceedings.mlr.press/v162/duan22d.html}, abstract = {Existing deep topic models are effective in capturing the latent semantic structures in textual data but usually rely on a plethora of documents. This is less than satisfactory in practical applications when only a limited amount of data is available. In this paper, we propose a novel framework that efficiently solves the problem of topic modeling under the small data regime. Specifically, the framework involves two innovations: a bi-level generative model that aims to exploit the task information to guide the document generation, and a topic meta-learner that strives to learn a group of global topic embeddings so that fast adaptation to the task-specific topic embeddings can be achieved with a few examples. We apply the proposed framework to a hierarchical embedded topic model and achieve better performance than various baseline models on diverse experiments, including few-shot topic discovery and few-shot document classification.} }
Endnote
%0 Conference Paper %T Bayesian Deep Embedding Topic Meta-Learner %A Zhibin Duan %A Yishi Xu %A Jianqiao Sun %A Bo Chen %A Wenchao Chen %A Chaojie Wang %A Mingyuan Zhou %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-duan22d %I PMLR %P 5659--5670 %U https://proceedings.mlr.press/v162/duan22d.html %V 162 %X Existing deep topic models are effective in capturing the latent semantic structures in textual data but usually rely on a plethora of documents. This is less than satisfactory in practical applications when only a limited amount of data is available. In this paper, we propose a novel framework that efficiently solves the problem of topic modeling under the small data regime. Specifically, the framework involves two innovations: a bi-level generative model that aims to exploit the task information to guide the document generation, and a topic meta-learner that strives to learn a group of global topic embeddings so that fast adaptation to the task-specific topic embeddings can be achieved with a few examples. We apply the proposed framework to a hierarchical embedded topic model and achieve better performance than various baseline models on diverse experiments, including few-shot topic discovery and few-shot document classification.
APA
Duan, Z., Xu, Y., Sun, J., Chen, B., Chen, W., Wang, C. & Zhou, M.. (2022). Bayesian Deep Embedding Topic Meta-Learner. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:5659-5670 Available from https://proceedings.mlr.press/v162/duan22d.html.

Related Material