Fast and Consistent Learning of Hidden Markov Models by Incorporating Non-Consecutive Correlations

Robert Mattila, Cristian Rojas, Eric Moulines, Vikram Krishnamurthy, Bo Wahlberg
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:6785-6796, 2020.

Abstract

Can the parameters of a hidden Markov model (HMM) be estimated from a single sweep through the observations – and additionally, without being trapped at a local optimum in the likelihood surface? That is the premise of recent method of moments algorithms devised for HMMs. In these, correlations between consecutive pair- or triplet-wise observations are empirically estimated and used to compute estimates of the HMM parameters. Albeit computationally very attractive, the main drawback is that by restricting to only low-order correlations in the data, information is being neglected which results in a loss of accuracy (compared to standard maximum likelihood schemes). In this paper, we propose extending these methods (both pair- and triplet-based) by also including non-consecutive correlations in a way which does not significantly increase the computational cost (which scales linearly with the number of additional lags included). We prove strong consistency of the new methods, and demonstrate an improved performance in numerical experiments on both synthetic and real-world financial time-series datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-mattila20a, title = {Fast and Consistent Learning of Hidden {M}arkov Models by Incorporating Non-Consecutive Correlations}, author = {Mattila, Robert and Rojas, Cristian and Moulines, Eric and Krishnamurthy, Vikram and Wahlberg, Bo}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {6785--6796}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/mattila20a/mattila20a.pdf}, url = {https://proceedings.mlr.press/v119/mattila20a.html}, abstract = {Can the parameters of a hidden Markov model (HMM) be estimated from a single sweep through the observations – and additionally, without being trapped at a local optimum in the likelihood surface? That is the premise of recent method of moments algorithms devised for HMMs. In these, correlations between consecutive pair- or triplet-wise observations are empirically estimated and used to compute estimates of the HMM parameters. Albeit computationally very attractive, the main drawback is that by restricting to only low-order correlations in the data, information is being neglected which results in a loss of accuracy (compared to standard maximum likelihood schemes). In this paper, we propose extending these methods (both pair- and triplet-based) by also including non-consecutive correlations in a way which does not significantly increase the computational cost (which scales linearly with the number of additional lags included). We prove strong consistency of the new methods, and demonstrate an improved performance in numerical experiments on both synthetic and real-world financial time-series datasets.} }
Endnote
%0 Conference Paper %T Fast and Consistent Learning of Hidden Markov Models by Incorporating Non-Consecutive Correlations %A Robert Mattila %A Cristian Rojas %A Eric Moulines %A Vikram Krishnamurthy %A Bo Wahlberg %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-mattila20a %I PMLR %P 6785--6796 %U https://proceedings.mlr.press/v119/mattila20a.html %V 119 %X Can the parameters of a hidden Markov model (HMM) be estimated from a single sweep through the observations – and additionally, without being trapped at a local optimum in the likelihood surface? That is the premise of recent method of moments algorithms devised for HMMs. In these, correlations between consecutive pair- or triplet-wise observations are empirically estimated and used to compute estimates of the HMM parameters. Albeit computationally very attractive, the main drawback is that by restricting to only low-order correlations in the data, information is being neglected which results in a loss of accuracy (compared to standard maximum likelihood schemes). In this paper, we propose extending these methods (both pair- and triplet-based) by also including non-consecutive correlations in a way which does not significantly increase the computational cost (which scales linearly with the number of additional lags included). We prove strong consistency of the new methods, and demonstrate an improved performance in numerical experiments on both synthetic and real-world financial time-series datasets.
APA
Mattila, R., Rojas, C., Moulines, E., Krishnamurthy, V. & Wahlberg, B.. (2020). Fast and Consistent Learning of Hidden Markov Models by Incorporating Non-Consecutive Correlations. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:6785-6796 Available from https://proceedings.mlr.press/v119/mattila20a.html.

Related Material