Data-Driven Near-Optimal Control of Nonlinear Systems Over Finite Horizon

Vasanth Reddy Baddam, Hoda Eldardiry, Almuatazbellah Boker
Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, PMLR 283:419-430, 2025.

Abstract

We employ reinforcement learning to address the problem of two-point boundary optimal control for nonlinear systems over a finite time horizon with unknown model dynamics. By leveraging techniques from singular perturbation theory, we decompose the finite-horizon control problem into two sub-problems, each defined over an infinite horizon. This decomposition eliminates the need to solve the time-varying Hamilton-Jacobi-Bellman (HJB) equation, significantly simplifying the process. Using a policy iteration method enabled by this decomposition, we learn the controller gains for each of the two sub-problems. The overall control strategy is then constructed by combining the solutions of these sub-problems. We demonstrate that the performance of the proposed closed-loop system asymptotically approaches the model-based optimal performance as the time horizon becomes large. Finally, we validate our approach through simulation scenarios, which provide strong support for the claims made in this paper.

Cite this Paper

BibTeX
@InProceedings{pmlr-v283-baddam25a, title = {Data-Driven Near-Optimal Control of Nonlinear Systems Over Finite Horizon}, author = {Baddam, Vasanth Reddy and Eldardiry, Hoda and Boker, Almuatazbellah}, booktitle = {Proceedings of the 7th Annual Learning for Dynamics \& Control Conference}, pages = {419--430}, year = {2025}, editor = {Ozay, Necmiye and Balzano, Laura and Panagou, Dimitra and Abate, Alessandro}, volume = {283}, series = {Proceedings of Machine Learning Research}, month = {04--06 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v283/main/assets/baddam25a/baddam25a.pdf}, url = {https://proceedings.mlr.press/v283/baddam25a.html}, abstract = {We employ reinforcement learning to address the problem of two-point boundary optimal control for nonlinear systems over a finite time horizon with unknown model dynamics. By leveraging techniques from singular perturbation theory, we decompose the finite-horizon control problem into two sub-problems, each defined over an infinite horizon. This decomposition eliminates the need to solve the time-varying Hamilton-Jacobi-Bellman (HJB) equation, significantly simplifying the process. Using a policy iteration method enabled by this decomposition, we learn the controller gains for each of the two sub-problems. The overall control strategy is then constructed by combining the solutions of these sub-problems. We demonstrate that the performance of the proposed closed-loop system asymptotically approaches the model-based optimal performance as the time horizon becomes large. Finally, we validate our approach through simulation scenarios, which provide strong support for the claims made in this paper.} }
Endnote
%0 Conference Paper %T Data-Driven Near-Optimal Control of Nonlinear Systems Over Finite Horizon %A Vasanth Reddy Baddam %A Hoda Eldardiry %A Almuatazbellah Boker %B Proceedings of the 7th Annual Learning for Dynamics \& Control Conference %C Proceedings of Machine Learning Research %D 2025 %E Necmiye Ozay %E Laura Balzano %E Dimitra Panagou %E Alessandro Abate %F pmlr-v283-baddam25a %I PMLR %P 419--430 %U https://proceedings.mlr.press/v283/baddam25a.html %V 283 %X We employ reinforcement learning to address the problem of two-point boundary optimal control for nonlinear systems over a finite time horizon with unknown model dynamics. By leveraging techniques from singular perturbation theory, we decompose the finite-horizon control problem into two sub-problems, each defined over an infinite horizon. This decomposition eliminates the need to solve the time-varying Hamilton-Jacobi-Bellman (HJB) equation, significantly simplifying the process. Using a policy iteration method enabled by this decomposition, we learn the controller gains for each of the two sub-problems. The overall control strategy is then constructed by combining the solutions of these sub-problems. We demonstrate that the performance of the proposed closed-loop system asymptotically approaches the model-based optimal performance as the time horizon becomes large. Finally, we validate our approach through simulation scenarios, which provide strong support for the claims made in this paper.
APA
Baddam, V.R., Eldardiry, H. & Boker, A.. (2025). Data-Driven Near-Optimal Control of Nonlinear Systems Over Finite Horizon. Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, in Proceedings of Machine Learning Research 283:419-430 Available from https://proceedings.mlr.press/v283/baddam25a.html.

Related Material