Bidirectional Learning for Timeseries Models with Hidden Units
[edit]
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:27112720, 2017.
Abstract
Hidden units can play essential roles in modeling timeseries having longterm dependency or onlinearity but make it difficult to learn associated parameters. Here we propose a way to learn such a timeseries model by training a backward model for the timereversed timeseries, where the backward model has a common set of parameters as the original (forward) model. Our key observation is that only a subset of the parameters is hard to learn, and that subset is complementary between the forward model and the backward model. By training both of the two models, we can effectively learn the values of the parameters that are hard to learn if only either of the two models is trained. We apply bidirectional learning to a dynamic Boltzmann machine extended with hidden units. Numerical experiments with synthetic and real datasets clearly demonstrate advantages of bidirectional learning.
Related Material


