Tensor Decompositions Meet Control Theory: Learning General Mixtures of Linear Dynamical Systems

Ainesh Bakshi, Allen Liu, Ankur Moitra, Morris Yau
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:1549-1563, 2023.

Abstract

Recently Chen and Poor initiated the study of learning mixtures of linear dynamical systems. While linear dynamical systems already have wide-ranging applications in modeling time-series data, using mixture models can lead to a better fit or even a richer understanding of underlying subpopulations represented in the data. In this work we give a new approach to learning mixtures of linear dynamical systems that is based on tensor decompositions. As a result, our algorithm succeeds without strong separation conditions on the components, and can be used to compete with the Bayes optimal clustering of the trajectories. Moreover our algorithm works in the challenging partially-observed setting. Our starting point is the simple but powerful observation that the classic Ho-Kalman algorithm is a relative of modern tensor decomposition methods for learning latent variable models. This gives us a playbook for how to extend it to work with more complicated generative models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-bakshi23a, title = {Tensor Decompositions Meet Control Theory: Learning General Mixtures of Linear Dynamical Systems}, author = {Bakshi, Ainesh and Liu, Allen and Moitra, Ankur and Yau, Morris}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {1549--1563}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/bakshi23a/bakshi23a.pdf}, url = {https://proceedings.mlr.press/v202/bakshi23a.html}, abstract = {Recently Chen and Poor initiated the study of learning mixtures of linear dynamical systems. While linear dynamical systems already have wide-ranging applications in modeling time-series data, using mixture models can lead to a better fit or even a richer understanding of underlying subpopulations represented in the data. In this work we give a new approach to learning mixtures of linear dynamical systems that is based on tensor decompositions. As a result, our algorithm succeeds without strong separation conditions on the components, and can be used to compete with the Bayes optimal clustering of the trajectories. Moreover our algorithm works in the challenging partially-observed setting. Our starting point is the simple but powerful observation that the classic Ho-Kalman algorithm is a relative of modern tensor decomposition methods for learning latent variable models. This gives us a playbook for how to extend it to work with more complicated generative models.} }
Endnote
%0 Conference Paper %T Tensor Decompositions Meet Control Theory: Learning General Mixtures of Linear Dynamical Systems %A Ainesh Bakshi %A Allen Liu %A Ankur Moitra %A Morris Yau %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-bakshi23a %I PMLR %P 1549--1563 %U https://proceedings.mlr.press/v202/bakshi23a.html %V 202 %X Recently Chen and Poor initiated the study of learning mixtures of linear dynamical systems. While linear dynamical systems already have wide-ranging applications in modeling time-series data, using mixture models can lead to a better fit or even a richer understanding of underlying subpopulations represented in the data. In this work we give a new approach to learning mixtures of linear dynamical systems that is based on tensor decompositions. As a result, our algorithm succeeds without strong separation conditions on the components, and can be used to compete with the Bayes optimal clustering of the trajectories. Moreover our algorithm works in the challenging partially-observed setting. Our starting point is the simple but powerful observation that the classic Ho-Kalman algorithm is a relative of modern tensor decomposition methods for learning latent variable models. This gives us a playbook for how to extend it to work with more complicated generative models.
APA
Bakshi, A., Liu, A., Moitra, A. & Yau, M.. (2023). Tensor Decompositions Meet Control Theory: Learning General Mixtures of Linear Dynamical Systems. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:1549-1563 Available from https://proceedings.mlr.press/v202/bakshi23a.html.

Related Material