Bidirectional Learning for Time-series Models with Hidden Units

Takayuki Osogami, Hiroshi Kajino, Taro Sekiyama
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:2711-2720, 2017.

Abstract

Hidden units can play essential roles in modeling time-series having long-term dependency or on-linearity but make it difficult to learn associated parameters. Here we propose a way to learn such a time-series model by training a backward model for the time-reversed time-series, where the backward model has a common set of parameters as the original (forward) model. Our key observation is that only a subset of the parameters is hard to learn, and that subset is complementary between the forward model and the backward model. By training both of the two models, we can effectively learn the values of the parameters that are hard to learn if only either of the two models is trained. We apply bidirectional learning to a dynamic Boltzmann machine extended with hidden units. Numerical experiments with synthetic and real datasets clearly demonstrate advantages of bidirectional learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-osogami17a, title = {Bidirectional Learning for Time-series Models with Hidden Units}, author = {Takayuki Osogami and Hiroshi Kajino and Taro Sekiyama}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {2711--2720}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/osogami17a/osogami17a.pdf}, url = {https://proceedings.mlr.press/v70/osogami17a.html}, abstract = {Hidden units can play essential roles in modeling time-series having long-term dependency or on-linearity but make it difficult to learn associated parameters. Here we propose a way to learn such a time-series model by training a backward model for the time-reversed time-series, where the backward model has a common set of parameters as the original (forward) model. Our key observation is that only a subset of the parameters is hard to learn, and that subset is complementary between the forward model and the backward model. By training both of the two models, we can effectively learn the values of the parameters that are hard to learn if only either of the two models is trained. We apply bidirectional learning to a dynamic Boltzmann machine extended with hidden units. Numerical experiments with synthetic and real datasets clearly demonstrate advantages of bidirectional learning.} }
Endnote
%0 Conference Paper %T Bidirectional Learning for Time-series Models with Hidden Units %A Takayuki Osogami %A Hiroshi Kajino %A Taro Sekiyama %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-osogami17a %I PMLR %P 2711--2720 %U https://proceedings.mlr.press/v70/osogami17a.html %V 70 %X Hidden units can play essential roles in modeling time-series having long-term dependency or on-linearity but make it difficult to learn associated parameters. Here we propose a way to learn such a time-series model by training a backward model for the time-reversed time-series, where the backward model has a common set of parameters as the original (forward) model. Our key observation is that only a subset of the parameters is hard to learn, and that subset is complementary between the forward model and the backward model. By training both of the two models, we can effectively learn the values of the parameters that are hard to learn if only either of the two models is trained. We apply bidirectional learning to a dynamic Boltzmann machine extended with hidden units. Numerical experiments with synthetic and real datasets clearly demonstrate advantages of bidirectional learning.
APA
Osogami, T., Kajino, H. & Sekiyama, T.. (2017). Bidirectional Learning for Time-series Models with Hidden Units. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:2711-2720 Available from https://proceedings.mlr.press/v70/osogami17a.html.

Related Material