Data-driven strategy synthesis for stochastic systems with unknown nonlinear disturbances

Ibon Gracia, Dimitris Boskos, Luca Laurenti, Morteza Lahijanian
Proceedings of the 6th Annual Learning for Dynamics & Control Conference, PMLR 242:1633-1645, 2024.

Abstract

In this paper, we introduce a data-driven framework for synthesis of provably-correct controllers for general nonlinear switched systems under complex specifications. The focus is on systems with unknown disturbances whose effects on the dynamics of the system is nonlinear. The specification is assumed to be given as linear temporal logic over finite traces (LTLf) formulas. Starting from observations of either the disturbance or the state of the system, we first learn an ambiguity set that contains the unknown distribution of the disturbances with a user-defined confidence. Next, we obtain a robust Markov decision process (RMDP) as a finite abstraction of the system. By composing the RMDP with the automaton obtained from the LTLf formula and performing optimal robust value iteration on the composed RMDP, we synthesize a strategy that yields a high probability that the uncertain system satisfies the specifications. Our empirical evaluations on systems with a wide variety of disturbances show that the strategies synthesized with our approach lead to high satisfaction probabilities and validate the theoretical guarantees.

Cite this Paper

BibTeX
@InProceedings{pmlr-v242-gracia24a, title = {Data-driven strategy synthesis for stochastic systems with unknown nonlinear disturbances}, author = {Gracia, Ibon and Boskos, Dimitris and Laurenti, Luca and Lahijanian, Morteza}, booktitle = {Proceedings of the 6th Annual Learning for Dynamics & Control Conference}, pages = {1633--1645}, year = {2024}, editor = {Abate, Alessandro and Cannon, Mark and Margellos, Kostas and Papachristodoulou, Antonis}, volume = {242}, series = {Proceedings of Machine Learning Research}, month = {15--17 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v242/gracia24a/gracia24a.pdf}, url = {https://proceedings.mlr.press/v242/gracia24a.html}, abstract = {In this paper, we introduce a data-driven framework for synthesis of provably-correct controllers for general nonlinear switched systems under complex specifications. The focus is on systems with unknown disturbances whose effects on the dynamics of the system is nonlinear. The specification is assumed to be given as linear temporal logic over finite traces (LTLf) formulas. Starting from observations of either the disturbance or the state of the system, we first learn an ambiguity set that contains the unknown distribution of the disturbances with a user-defined confidence. Next, we obtain a robust Markov decision process (RMDP) as a finite abstraction of the system. By composing the RMDP with the automaton obtained from the LTLf formula and performing optimal robust value iteration on the composed RMDP, we synthesize a strategy that yields a high probability that the uncertain system satisfies the specifications. Our empirical evaluations on systems with a wide variety of disturbances show that the strategies synthesized with our approach lead to high satisfaction probabilities and validate the theoretical guarantees.} }
Endnote
%0 Conference Paper %T Data-driven strategy synthesis for stochastic systems with unknown nonlinear disturbances %A Ibon Gracia %A Dimitris Boskos %A Luca Laurenti %A Morteza Lahijanian %B Proceedings of the 6th Annual Learning for Dynamics & Control Conference %C Proceedings of Machine Learning Research %D 2024 %E Alessandro Abate %E Mark Cannon %E Kostas Margellos %E Antonis Papachristodoulou %F pmlr-v242-gracia24a %I PMLR %P 1633--1645 %U https://proceedings.mlr.press/v242/gracia24a.html %V 242 %X In this paper, we introduce a data-driven framework for synthesis of provably-correct controllers for general nonlinear switched systems under complex specifications. The focus is on systems with unknown disturbances whose effects on the dynamics of the system is nonlinear. The specification is assumed to be given as linear temporal logic over finite traces (LTLf) formulas. Starting from observations of either the disturbance or the state of the system, we first learn an ambiguity set that contains the unknown distribution of the disturbances with a user-defined confidence. Next, we obtain a robust Markov decision process (RMDP) as a finite abstraction of the system. By composing the RMDP with the automaton obtained from the LTLf formula and performing optimal robust value iteration on the composed RMDP, we synthesize a strategy that yields a high probability that the uncertain system satisfies the specifications. Our empirical evaluations on systems with a wide variety of disturbances show that the strategies synthesized with our approach lead to high satisfaction probabilities and validate the theoretical guarantees.
APA
Gracia, I., Boskos, D., Laurenti, L. & Lahijanian, M.. (2024). Data-driven strategy synthesis for stochastic systems with unknown nonlinear disturbances. Proceedings of the 6th Annual Learning for Dynamics & Control Conference, in Proceedings of Machine Learning Research 242:1633-1645 Available from https://proceedings.mlr.press/v242/gracia24a.html.

Related Material