A Hybrid Neural Network-Latent Topic Model

Li Wan, Leo Zhu, Rob Fergus
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:1287-1294, 2012.

Abstract

This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a low dimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-wan12, title = {A Hybrid Neural Network-Latent Topic Model}, author = {Wan, Li and Zhu, Leo and Fergus, Rob}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {1287--1294}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/wan12/wan12.pdf}, url = {https://proceedings.mlr.press/v22/wan12.html}, abstract = {This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a low dimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods.} }
Endnote
%0 Conference Paper %T A Hybrid Neural Network-Latent Topic Model %A Li Wan %A Leo Zhu %A Rob Fergus %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-wan12 %I PMLR %P 1287--1294 %U https://proceedings.mlr.press/v22/wan12.html %V 22 %X This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a low dimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods.
RIS
TY - CPAPER TI - A Hybrid Neural Network-Latent Topic Model AU - Li Wan AU - Leo Zhu AU - Rob Fergus BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-wan12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 1287 EP - 1294 L1 - http://proceedings.mlr.press/v22/wan12/wan12.pdf UR - https://proceedings.mlr.press/v22/wan12.html AB - This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a low dimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods. ER -
APA
Wan, L., Zhu, L. & Fergus, R.. (2012). A Hybrid Neural Network-Latent Topic Model. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:1287-1294 Available from https://proceedings.mlr.press/v22/wan12.html.

Related Material