Factored Temporal Sigmoid Belief Networks for Sequence Learning

Jiaming Song, Zhe Gan, Lawrence Carin
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:1272-1281, 2016.

Abstract

Deep conditional generative models are developed to simultaneously learn the temporal dependencies of multiple sequences. The model is designed by introducing a three-way weight tensor to capture the multiplicative interactions between side information and sequences. The proposed model builds on the Temporal Sigmoid Belief Network (TSBN), a sequential stack of Sigmoid Belief Networks (SBNs). The transition matrices are further factored to reduce the number of parameters and improve generalization. When side information is not available, a general framework for semi-supervised learning based on the proposed model is constituted, allowing robust sequence classification. Experimental results show that the proposed approach achieves state-of-the-art predictive and classification performance on sequential data, and has the capacity to synthesize sequences, with controlled style transitioning and blending.

Cite this Paper

BibTeX
@InProceedings{pmlr-v48-songa16, title = {Factored Temporal Sigmoid Belief Networks for Sequence Learning}, author = {Song, Jiaming and Gan, Zhe and Carin, Lawrence}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {1272--1281}, year = {2016}, editor = {Balcan, Maria Florina and Weinberger, Kilian Q.}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/songa16.pdf}, url = {https://proceedings.mlr.press/v48/songa16.html}, abstract = {Deep conditional generative models are developed to simultaneously learn the temporal dependencies of multiple sequences. The model is designed by introducing a three-way weight tensor to capture the multiplicative interactions between side information and sequences. The proposed model builds on the Temporal Sigmoid Belief Network (TSBN), a sequential stack of Sigmoid Belief Networks (SBNs). The transition matrices are further factored to reduce the number of parameters and improve generalization. When side information is not available, a general framework for semi-supervised learning based on the proposed model is constituted, allowing robust sequence classification. Experimental results show that the proposed approach achieves state-of-the-art predictive and classification performance on sequential data, and has the capacity to synthesize sequences, with controlled style transitioning and blending.} }
Endnote
%0 Conference Paper %T Factored Temporal Sigmoid Belief Networks for Sequence Learning %A Jiaming Song %A Zhe Gan %A Lawrence Carin %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-songa16 %I PMLR %P 1272--1281 %U https://proceedings.mlr.press/v48/songa16.html %V 48 %X Deep conditional generative models are developed to simultaneously learn the temporal dependencies of multiple sequences. The model is designed by introducing a three-way weight tensor to capture the multiplicative interactions between side information and sequences. The proposed model builds on the Temporal Sigmoid Belief Network (TSBN), a sequential stack of Sigmoid Belief Networks (SBNs). The transition matrices are further factored to reduce the number of parameters and improve generalization. When side information is not available, a general framework for semi-supervised learning based on the proposed model is constituted, allowing robust sequence classification. Experimental results show that the proposed approach achieves state-of-the-art predictive and classification performance on sequential data, and has the capacity to synthesize sequences, with controlled style transitioning and blending.
RIS
TY - CPAPER TI - Factored Temporal Sigmoid Belief Networks for Sequence Learning AU - Jiaming Song AU - Zhe Gan AU - Lawrence Carin BT - Proceedings of The 33rd International Conference on Machine Learning DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-songa16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 48 SP - 1272 EP - 1281 L1 - http://proceedings.mlr.press/v48/songa16.pdf UR - https://proceedings.mlr.press/v48/songa16.html AB - Deep conditional generative models are developed to simultaneously learn the temporal dependencies of multiple sequences. The model is designed by introducing a three-way weight tensor to capture the multiplicative interactions between side information and sequences. The proposed model builds on the Temporal Sigmoid Belief Network (TSBN), a sequential stack of Sigmoid Belief Networks (SBNs). The transition matrices are further factored to reduce the number of parameters and improve generalization. When side information is not available, a general framework for semi-supervised learning based on the proposed model is constituted, allowing robust sequence classification. Experimental results show that the proposed approach achieves state-of-the-art predictive and classification performance on sequential data, and has the capacity to synthesize sequences, with controlled style transitioning and blending. ER -
APA
Song, J., Gan, Z. & Carin, L.. (2016). Factored Temporal Sigmoid Belief Networks for Sequence Learning. Proceedings of The 33rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 48:1272-1281 Available from https://proceedings.mlr.press/v48/songa16.html.

Related Material