Regret Analysis of Online LQR Control via Trajectory Prediction and Tracking

Yitian Chen, Timothy L Molloy, Tyler Summers, Iman Shames
Proceedings of The 5th Annual Learning for Dynamics and Control Conference, PMLR 211:248-258, 2023.

Abstract

In this paper, we propose and analyse a new method for online linear quadratic regulator (LQR) control with a priori unknown time-varying cost matrices. The cost matrices are revealed sequentially with the potential for future values to be previewed over a short window. Our novel method involves using the available cost matrices to predict the optimal trajectory, and a tracking controller to drive the system towards it. We adopted the notion of dynamic regret to measure the performance of this proposed online LQR control method, with our main result being that the (dynamic) regret of our method is upper bounded by a constant. Moreover, the regret upper bound decays exponentially with the preview window length, and is extendable to systems with disturbances. We show in simulations that our proposed method offers improved performance compared to other previously proposed online LQR methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v211-chen23a, title = {Regret Analysis of Online LQR Control via Trajectory Prediction and Tracking}, author = {Chen, Yitian and Molloy, Timothy L and Summers, Tyler and Shames, Iman}, booktitle = {Proceedings of The 5th Annual Learning for Dynamics and Control Conference}, pages = {248--258}, year = {2023}, editor = {Matni, Nikolai and Morari, Manfred and Pappas, George J.}, volume = {211}, series = {Proceedings of Machine Learning Research}, month = {15--16 Jun}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v211/chen23a/chen23a.pdf}, url = {https://proceedings.mlr.press/v211/chen23a.html}, abstract = {In this paper, we propose and analyse a new method for online linear quadratic regulator (LQR) control with a priori unknown time-varying cost matrices. The cost matrices are revealed sequentially with the potential for future values to be previewed over a short window. Our novel method involves using the available cost matrices to predict the optimal trajectory, and a tracking controller to drive the system towards it. We adopted the notion of dynamic regret to measure the performance of this proposed online LQR control method, with our main result being that the (dynamic) regret of our method is upper bounded by a constant. Moreover, the regret upper bound decays exponentially with the preview window length, and is extendable to systems with disturbances. We show in simulations that our proposed method offers improved performance compared to other previously proposed online LQR methods.} }
Endnote
%0 Conference Paper %T Regret Analysis of Online LQR Control via Trajectory Prediction and Tracking %A Yitian Chen %A Timothy L Molloy %A Tyler Summers %A Iman Shames %B Proceedings of The 5th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2023 %E Nikolai Matni %E Manfred Morari %E George J. Pappas %F pmlr-v211-chen23a %I PMLR %P 248--258 %U https://proceedings.mlr.press/v211/chen23a.html %V 211 %X In this paper, we propose and analyse a new method for online linear quadratic regulator (LQR) control with a priori unknown time-varying cost matrices. The cost matrices are revealed sequentially with the potential for future values to be previewed over a short window. Our novel method involves using the available cost matrices to predict the optimal trajectory, and a tracking controller to drive the system towards it. We adopted the notion of dynamic regret to measure the performance of this proposed online LQR control method, with our main result being that the (dynamic) regret of our method is upper bounded by a constant. Moreover, the regret upper bound decays exponentially with the preview window length, and is extendable to systems with disturbances. We show in simulations that our proposed method offers improved performance compared to other previously proposed online LQR methods.
APA
Chen, Y., Molloy, T.L., Summers, T. & Shames, I.. (2023). Regret Analysis of Online LQR Control via Trajectory Prediction and Tracking. Proceedings of The 5th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 211:248-258 Available from https://proceedings.mlr.press/v211/chen23a.html.

Related Material