A Hybrid Neural Network-Latent Topic Model

Li Wan, Leo Zhu, Rob Fergus
; Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:1287-1294, 2012.

Abstract

This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a low dimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods.

Cite this Paper


BibTeX
@InProceedings{pmlr-v22-wan12, title = {A Hybrid Neural Network-Latent Topic Model}, author = {Li Wan and Leo Zhu and Rob Fergus}, pages = {1287--1294}, year = {2012}, editor = {Neil D. Lawrence and Mark Girolami}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/wan12/wan12.pdf}, url = {http://proceedings.mlr.press/v22/wan12.html}, abstract = {This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a low dimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods.} }
Endnote
%0 Conference Paper %T A Hybrid Neural Network-Latent Topic Model %A Li Wan %A Leo Zhu %A Rob Fergus %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-wan12 %I PMLR %J Proceedings of Machine Learning Research %P 1287--1294 %U http://proceedings.mlr.press %V 22 %W PMLR %X This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a low dimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods.
RIS
TY - CPAPER TI - A Hybrid Neural Network-Latent Topic Model AU - Li Wan AU - Leo Zhu AU - Rob Fergus BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics PY - 2012/03/21 DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-wan12 PB - PMLR SP - 1287 DP - PMLR EP - 1294 L1 - http://proceedings.mlr.press/v22/wan12/wan12.pdf UR - http://proceedings.mlr.press/v22/wan12.html AB - This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a low dimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods. ER -
APA
Wan, L., Zhu, L. & Fergus, R.. (2012). A Hybrid Neural Network-Latent Topic Model. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in PMLR 22:1287-1294

Related Material