Data-driven optimal control of unknown nonlinear dynamical systems using the Koopman operator

Zhexuan Zeng, Ruikun Zhou, Yiming Meng, Jun Liu
Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, PMLR 283:1127-1139, 2025.

Abstract

Nonlinear optimal control is vital for numerous applications but remains challenging for unknown systems due to the difficulties in accurately modelling dynamics and handling computational demands, particularly in high-dimensional settings. This work develops a theoretically certifiable framework that integrates a modified Koopman operator approach with model-based reinforcement learning to address these challenges. By relaxing the requirements on observable functions, our method incorporates nonlinear terms involving both states and control inputs, significantly enhancing system identification accuracy. Moreover, by leveraging the power of neural networks to solve partial differential equations (PDEs), our approach is able to achieving stabilizing control for high-dimensional dynamical systems, up to 9-dimensional. The learned value function and control laws are proven to converge to those of the true system at each iteration. Additionally, the accumulated cost of the learned control closely approximates that of the true system, with errors ranging from $10^{-5}$ to $10^{-3}$.

Cite this Paper

BibTeX
@InProceedings{pmlr-v283-zeng25a, title = {Data-driven optimal control of unknown nonlinear dynamical systems using the Koopman operator}, author = {Zeng, Zhexuan and Zhou, Ruikun and Meng, Yiming and Liu, Jun}, booktitle = {Proceedings of the 7th Annual Learning for Dynamics \& Control Conference}, pages = {1127--1139}, year = {2025}, editor = {Ozay, Necmiye and Balzano, Laura and Panagou, Dimitra and Abate, Alessandro}, volume = {283}, series = {Proceedings of Machine Learning Research}, month = {04--06 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v283/main/assets/zeng25a/zeng25a.pdf}, url = {https://proceedings.mlr.press/v283/zeng25a.html}, abstract = {Nonlinear optimal control is vital for numerous applications but remains challenging for unknown systems due to the difficulties in accurately modelling dynamics and handling computational demands, particularly in high-dimensional settings. This work develops a theoretically certifiable framework that integrates a modified Koopman operator approach with model-based reinforcement learning to address these challenges. By relaxing the requirements on observable functions, our method incorporates nonlinear terms involving both states and control inputs, significantly enhancing system identification accuracy. Moreover, by leveraging the power of neural networks to solve partial differential equations (PDEs), our approach is able to achieving stabilizing control for high-dimensional dynamical systems, up to 9-dimensional. The learned value function and control laws are proven to converge to those of the true system at each iteration. Additionally, the accumulated cost of the learned control closely approximates that of the true system, with errors ranging from $10^{-5}$ to $10^{-3}$.} }
Endnote
%0 Conference Paper %T Data-driven optimal control of unknown nonlinear dynamical systems using the Koopman operator %A Zhexuan Zeng %A Ruikun Zhou %A Yiming Meng %A Jun Liu %B Proceedings of the 7th Annual Learning for Dynamics \& Control Conference %C Proceedings of Machine Learning Research %D 2025 %E Necmiye Ozay %E Laura Balzano %E Dimitra Panagou %E Alessandro Abate %F pmlr-v283-zeng25a %I PMLR %P 1127--1139 %U https://proceedings.mlr.press/v283/zeng25a.html %V 283 %X Nonlinear optimal control is vital for numerous applications but remains challenging for unknown systems due to the difficulties in accurately modelling dynamics and handling computational demands, particularly in high-dimensional settings. This work develops a theoretically certifiable framework that integrates a modified Koopman operator approach with model-based reinforcement learning to address these challenges. By relaxing the requirements on observable functions, our method incorporates nonlinear terms involving both states and control inputs, significantly enhancing system identification accuracy. Moreover, by leveraging the power of neural networks to solve partial differential equations (PDEs), our approach is able to achieving stabilizing control for high-dimensional dynamical systems, up to 9-dimensional. The learned value function and control laws are proven to converge to those of the true system at each iteration. Additionally, the accumulated cost of the learned control closely approximates that of the true system, with errors ranging from $10^{-5}$ to $10^{-3}$.
APA
Zeng, Z., Zhou, R., Meng, Y. & Liu, J.. (2025). Data-driven optimal control of unknown nonlinear dynamical systems using the Koopman operator. Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, in Proceedings of Machine Learning Research 283:1127-1139 Available from https://proceedings.mlr.press/v283/zeng25a.html.

Related Material