Adaptive Risk Sensitive Model Predictive Control with Stochastic Search

Ziyi Wang, Oswin So, Keuntaek Lee, Evangelos A. Theodorou
Proceedings of the 3rd Conference on Learning for Dynamics and Control, PMLR 144:510-522, 2021.

Abstract

We present a general framework for optimizing the Conditional Value-at-Risk for dynamical systems using stochastic search. The framework is capable of handling the uncertainty from the initial condition, stochastic dynamics, and uncertain parameters in the model. The algorithm is compared against a risk-sensitive distributional reinforcement learning framework and demonstrates improved performance on a pendulum and cartpole with stochastic dynamics. We also showcase the applicability of the framework to robotics as an adaptive risk-sensitive controller by optimizing with respect to the fully nonlinear belief provided by a particle filter on a pendulum, cartpole, and quadcopter in simulation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v144-wang21b, title = {Adaptive Risk Sensitive Model Predictive Control with Stochastic Search}, author = {Wang, Ziyi and So, Oswin and Lee, Keuntaek and Theodorou, Evangelos A.}, booktitle = {Proceedings of the 3rd Conference on Learning for Dynamics and Control}, pages = {510--522}, year = {2021}, editor = {Jadbabaie, Ali and Lygeros, John and Pappas, George J. and A. Parrilo, Pablo and Recht, Benjamin and Tomlin, Claire J. and Zeilinger, Melanie N.}, volume = {144}, series = {Proceedings of Machine Learning Research}, month = {07 -- 08 June}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v144/wang21b/wang21b.pdf}, url = {https://proceedings.mlr.press/v144/wang21b.html}, abstract = {We present a general framework for optimizing the Conditional Value-at-Risk for dynamical systems using stochastic search. The framework is capable of handling the uncertainty from the initial condition, stochastic dynamics, and uncertain parameters in the model. The algorithm is compared against a risk-sensitive distributional reinforcement learning framework and demonstrates improved performance on a pendulum and cartpole with stochastic dynamics. We also showcase the applicability of the framework to robotics as an adaptive risk-sensitive controller by optimizing with respect to the fully nonlinear belief provided by a particle filter on a pendulum, cartpole, and quadcopter in simulation.} }
Endnote
%0 Conference Paper %T Adaptive Risk Sensitive Model Predictive Control with Stochastic Search %A Ziyi Wang %A Oswin So %A Keuntaek Lee %A Evangelos A. Theodorou %B Proceedings of the 3rd Conference on Learning for Dynamics and Control %C Proceedings of Machine Learning Research %D 2021 %E Ali Jadbabaie %E John Lygeros %E George J. Pappas %E Pablo A. Parrilo %E Benjamin Recht %E Claire J. Tomlin %E Melanie N. Zeilinger %F pmlr-v144-wang21b %I PMLR %P 510--522 %U https://proceedings.mlr.press/v144/wang21b.html %V 144 %X We present a general framework for optimizing the Conditional Value-at-Risk for dynamical systems using stochastic search. The framework is capable of handling the uncertainty from the initial condition, stochastic dynamics, and uncertain parameters in the model. The algorithm is compared against a risk-sensitive distributional reinforcement learning framework and demonstrates improved performance on a pendulum and cartpole with stochastic dynamics. We also showcase the applicability of the framework to robotics as an adaptive risk-sensitive controller by optimizing with respect to the fully nonlinear belief provided by a particle filter on a pendulum, cartpole, and quadcopter in simulation.
APA
Wang, Z., So, O., Lee, K. & Theodorou, E.A.. (2021). Adaptive Risk Sensitive Model Predictive Control with Stochastic Search. Proceedings of the 3rd Conference on Learning for Dynamics and Control, in Proceedings of Machine Learning Research 144:510-522 Available from https://proceedings.mlr.press/v144/wang21b.html.

Related Material