Adaptive Control of Positive Systems with Application to Learning SSP

Fethi Bencherki, Anders Rantzer
Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, PMLR 283:660-672, 2025.

Abstract

An adaptive controller is proposed and analyzed for the class of infinite-horizon optimal control problems in positive linear systems presented in (Ohlin et al., 2024b). This controller is derived from the solution of a “data-driven algebraic equation” constructed using the model-free Bellman equation from Q-learning. The equation is driven by data correlation matrices that do not scale with the number of data points, enabling efficient online implementation. Consequently, a sufficient condition guaranteeing stability and robustness to unmodeled dynamics is established. The derived results also provide a quantitative characterization of the interplay between excitation level and robustness to unmodeled dynamics. The class of optimal control problems considered here is equivalent to Stochastic Shortest Path (SSP) problems, allowing for a performance comparison between the proposed adaptive policy and model-free algorithms for learning the stochastic shortest path, as demonstrated in the numerical experiment.

Cite this Paper

BibTeX
@InProceedings{pmlr-v283-bencherki25a, title = {Adaptive Control of Positive Systems with Application to Learning SSP}, author = {Bencherki, Fethi and Rantzer, Anders}, booktitle = {Proceedings of the 7th Annual Learning for Dynamics \& Control Conference}, pages = {660--672}, year = {2025}, editor = {Ozay, Necmiye and Balzano, Laura and Panagou, Dimitra and Abate, Alessandro}, volume = {283}, series = {Proceedings of Machine Learning Research}, month = {04--06 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v283/main/assets/bencherki25a/bencherki25a.pdf}, url = {https://proceedings.mlr.press/v283/bencherki25a.html}, abstract = {An adaptive controller is proposed and analyzed for the class of infinite-horizon optimal control problems in positive linear systems presented in (Ohlin et al., 2024b). This controller is derived from the solution of a “data-driven algebraic equation” constructed using the model-free Bellman equation from Q-learning. The equation is driven by data correlation matrices that do not scale with the number of data points, enabling efficient online implementation. Consequently, a sufficient condition guaranteeing stability and robustness to unmodeled dynamics is established. The derived results also provide a quantitative characterization of the interplay between excitation level and robustness to unmodeled dynamics. The class of optimal control problems considered here is equivalent to Stochastic Shortest Path (SSP) problems, allowing for a performance comparison between the proposed adaptive policy and model-free algorithms for learning the stochastic shortest path, as demonstrated in the numerical experiment.} }
Endnote
%0 Conference Paper %T Adaptive Control of Positive Systems with Application to Learning SSP %A Fethi Bencherki %A Anders Rantzer %B Proceedings of the 7th Annual Learning for Dynamics \& Control Conference %C Proceedings of Machine Learning Research %D 2025 %E Necmiye Ozay %E Laura Balzano %E Dimitra Panagou %E Alessandro Abate %F pmlr-v283-bencherki25a %I PMLR %P 660--672 %U https://proceedings.mlr.press/v283/bencherki25a.html %V 283 %X An adaptive controller is proposed and analyzed for the class of infinite-horizon optimal control problems in positive linear systems presented in (Ohlin et al., 2024b). This controller is derived from the solution of a “data-driven algebraic equation” constructed using the model-free Bellman equation from Q-learning. The equation is driven by data correlation matrices that do not scale with the number of data points, enabling efficient online implementation. Consequently, a sufficient condition guaranteeing stability and robustness to unmodeled dynamics is established. The derived results also provide a quantitative characterization of the interplay between excitation level and robustness to unmodeled dynamics. The class of optimal control problems considered here is equivalent to Stochastic Shortest Path (SSP) problems, allowing for a performance comparison between the proposed adaptive policy and model-free algorithms for learning the stochastic shortest path, as demonstrated in the numerical experiment.
APA
Bencherki, F. & Rantzer, A.. (2025). Adaptive Control of Positive Systems with Application to Learning SSP. Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, in Proceedings of Machine Learning Research 283:660-672 Available from https://proceedings.mlr.press/v283/bencherki25a.html.

Related Material