Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems

Sahin Lale, Kamyar Azizzadenesheli, Babak Hassibi, Anima Anandkumar
Proceedings of the 3rd Conference on Learning for Dynamics and Control, PMLR 144:967-979, 2021.

Abstract

Autoregressive exogenous (ARX) systems are the general class of input-output dynamical system used for modeling stochastic linear dynamical system (LDS) including partially observable LDS such as LQG systems. In this work, we study the problem of system identification and adaptive control of unknown ARX systems. We provide finite-time learning guarantees for the ARX systems under both open-loop and closed-loop data collection. Using these guarantees, we design adaptive control algorithms for unknown ARX systems with arbitrary strongly convex or non-strongly convex quadratic regulating costs. Under strongly convex cost functions, we design an adaptive control algorithm based on online gradient descent to design and update the controllers that are constructed via a convex controller reparametrization. We show that our algorithm has $\Tilde{O}(\sqrt{T})$ regret via explore and commit approach and if the model estimates are updated in epochs using closed-loop data collection, it attains the optimal regret of $\text{polylog}(T)$ after $T$ time-steps of interaction. For the case of non-strongly convex quadratic cost functions, we propose an adaptive control algorithm that deploys the optimism in the face of uncertainty principle to design the controller. In this setting, we show that the explore and commit approach has a regret upper bound of $\Tilde{O}(T^{2/3})$, and the adaptive control with continuous model estimate updates attains $\Tilde{O}(\sqrt{T})$ regret after $T$ time-steps.

Cite this Paper

BibTeX
@InProceedings{pmlr-v144-lale21b, title = {Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems}, author = {Lale, Sahin and Azizzadenesheli, Kamyar and Hassibi, Babak and Anandkumar, Anima}, booktitle = {Proceedings of the 3rd Conference on Learning for Dynamics and Control}, pages = {967--979}, year = {2021}, editor = {Jadbabaie, Ali and Lygeros, John and Pappas, George J. and A. Parrilo, Pablo and Recht, Benjamin and Tomlin, Claire J. and Zeilinger, Melanie N.}, volume = {144}, series = {Proceedings of Machine Learning Research}, month = {07 -- 08 June}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v144/lale21b/lale21b.pdf}, url = {https://proceedings.mlr.press/v144/lale21b.html}, abstract = {Autoregressive exogenous (ARX) systems are the general class of input-output dynamical system used for modeling stochastic linear dynamical system (LDS) including partially observable LDS such as LQG systems. In this work, we study the problem of system identification and adaptive control of unknown ARX systems. We provide finite-time learning guarantees for the ARX systems under both open-loop and closed-loop data collection. Using these guarantees, we design adaptive control algorithms for unknown ARX systems with arbitrary strongly convex or non-strongly convex quadratic regulating costs. Under strongly convex cost functions, we design an adaptive control algorithm based on online gradient descent to design and update the controllers that are constructed via a convex controller reparametrization. We show that our algorithm has $\Tilde{O}(\sqrt{T})$ regret via explore and commit approach and if the model estimates are updated in epochs using closed-loop data collection, it attains the optimal regret of $\text{polylog}(T)$ after $T$ time-steps of interaction. For the case of non-strongly convex quadratic cost functions, we propose an adaptive control algorithm that deploys the optimism in the face of uncertainty principle to design the controller. In this setting, we show that the explore and commit approach has a regret upper bound of $\Tilde{O}(T^{2/3})$, and the adaptive control with continuous model estimate updates attains $\Tilde{O}(\sqrt{T})$ regret after $T$ time-steps. } }
Endnote
%0 Conference Paper %T Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems %A Sahin Lale %A Kamyar Azizzadenesheli %A Babak Hassibi %A Anima Anandkumar %B Proceedings of the 3rd Conference on Learning for Dynamics and Control %C Proceedings of Machine Learning Research %D 2021 %E Ali Jadbabaie %E John Lygeros %E George J. Pappas %E Pablo A. Parrilo %E Benjamin Recht %E Claire J. Tomlin %E Melanie N. Zeilinger %F pmlr-v144-lale21b %I PMLR %P 967--979 %U https://proceedings.mlr.press/v144/lale21b.html %V 144 %X Autoregressive exogenous (ARX) systems are the general class of input-output dynamical system used for modeling stochastic linear dynamical system (LDS) including partially observable LDS such as LQG systems. In this work, we study the problem of system identification and adaptive control of unknown ARX systems. We provide finite-time learning guarantees for the ARX systems under both open-loop and closed-loop data collection. Using these guarantees, we design adaptive control algorithms for unknown ARX systems with arbitrary strongly convex or non-strongly convex quadratic regulating costs. Under strongly convex cost functions, we design an adaptive control algorithm based on online gradient descent to design and update the controllers that are constructed via a convex controller reparametrization. We show that our algorithm has $\Tilde{O}(\sqrt{T})$ regret via explore and commit approach and if the model estimates are updated in epochs using closed-loop data collection, it attains the optimal regret of $\text{polylog}(T)$ after $T$ time-steps of interaction. For the case of non-strongly convex quadratic cost functions, we propose an adaptive control algorithm that deploys the optimism in the face of uncertainty principle to design the controller. In this setting, we show that the explore and commit approach has a regret upper bound of $\Tilde{O}(T^{2/3})$, and the adaptive control with continuous model estimate updates attains $\Tilde{O}(\sqrt{T})$ regret after $T$ time-steps.
APA
Lale, S., Azizzadenesheli, K., Hassibi, B. & Anandkumar, A.. (2021). Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems. Proceedings of the 3rd Conference on Learning for Dynamics and Control, in Proceedings of Machine Learning Research 144:967-979 Available from https://proceedings.mlr.press/v144/lale21b.html.

Related Material