Safe Reinforcement Learning Using Robust Action Governor

Yutong Li, Nan Li, H. Eric Tseng, Anouck Girard, Dimitar Filev, Ilya Kolmanovsky
Proceedings of the 3rd Conference on Learning for Dynamics and Control, PMLR 144:1093-1104, 2021.

Abstract

Reinforcement Learning (RL) is essentially a trial-and-error learning procedure which may cause unsafe behavior during the exploration-and-exploitation process. This hinders the application of RL to real-world control problems, especially to those for safety-critical systems. In this paper, we introduce a framework for safe RL that is based on integration of an RL algorithm with an add-on safety supervision module, called the Robust Action Governor (RAG), which exploits set-theoretic techniques and online optimization to manage safety-related requirements during learning. We illustrate this proposed safe RL framework through an application to automotive adaptive cruise control.

Cite this Paper

BibTeX
@InProceedings{pmlr-v144-li21b, title = {Safe Reinforcement Learning Using Robust Action Governor}, author = {Li, Yutong and Li, Nan and Tseng, H. Eric and Girard, Anouck and Filev, Dimitar and Kolmanovsky, Ilya}, booktitle = {Proceedings of the 3rd Conference on Learning for Dynamics and Control}, pages = {1093--1104}, year = {2021}, editor = {Jadbabaie, Ali and Lygeros, John and Pappas, George J. and A. Parrilo, Pablo and Recht, Benjamin and Tomlin, Claire J. and Zeilinger, Melanie N.}, volume = {144}, series = {Proceedings of Machine Learning Research}, month = {07 -- 08 June}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v144/li21b/li21b.pdf}, url = {https://proceedings.mlr.press/v144/li21b.html}, abstract = {Reinforcement Learning (RL) is essentially a trial-and-error learning procedure which may cause unsafe behavior during the exploration-and-exploitation process. This hinders the application of RL to real-world control problems, especially to those for safety-critical systems. In this paper, we introduce a framework for safe RL that is based on integration of an RL algorithm with an add-on safety supervision module, called the Robust Action Governor (RAG), which exploits set-theoretic techniques and online optimization to manage safety-related requirements during learning. We illustrate this proposed safe RL framework through an application to automotive adaptive cruise control.} }
Endnote
%0 Conference Paper %T Safe Reinforcement Learning Using Robust Action Governor %A Yutong Li %A Nan Li %A H. Eric Tseng %A Anouck Girard %A Dimitar Filev %A Ilya Kolmanovsky %B Proceedings of the 3rd Conference on Learning for Dynamics and Control %C Proceedings of Machine Learning Research %D 2021 %E Ali Jadbabaie %E John Lygeros %E George J. Pappas %E Pablo A. Parrilo %E Benjamin Recht %E Claire J. Tomlin %E Melanie N. Zeilinger %F pmlr-v144-li21b %I PMLR %P 1093--1104 %U https://proceedings.mlr.press/v144/li21b.html %V 144 %X Reinforcement Learning (RL) is essentially a trial-and-error learning procedure which may cause unsafe behavior during the exploration-and-exploitation process. This hinders the application of RL to real-world control problems, especially to those for safety-critical systems. In this paper, we introduce a framework for safe RL that is based on integration of an RL algorithm with an add-on safety supervision module, called the Robust Action Governor (RAG), which exploits set-theoretic techniques and online optimization to manage safety-related requirements during learning. We illustrate this proposed safe RL framework through an application to automotive adaptive cruise control.
APA
Li, Y., Li, N., Tseng, H.E., Girard, A., Filev, D. & Kolmanovsky, I.. (2021). Safe Reinforcement Learning Using Robust Action Governor. Proceedings of the 3rd Conference on Learning for Dynamics and Control, in Proceedings of Machine Learning Research 144:1093-1104 Available from https://proceedings.mlr.press/v144/li21b.html.

Related Material