Variational Optimization Based Reinforcement Learning for Infinite Dimensional Stochastic Systems

Ethan N. Evans, Marcus A. Periera, George I. Boutselis, Evangelos A. Theodorou
Proceedings of the Conference on Robot Learning, PMLR 100:1231-1246, 2020.

Abstract

Systems involving Partial Differential Equations (PDEs) have recently become more popular among the machine learning community. However prior methods usually treat infinite dimensional problems in finite dimensions with Reduced Order Models. This leads to committing to specific approximation schemes and subsequent derivation of control laws. Additionally, prior work does not consider spatio-temporal descriptions of noise that realistically represent the stochastic nature of physical systems. In this paper we suggest a new reinforcement learning framework that is mostly model-free for Stochastic PDEs with additive spacetime noise, based on variational optimization in infinite dimensions. In addition, our algorithm incorporates sparse representations that allow for efficient learning of feedback policies in high dimensions. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.

Cite this Paper

BibTeX
@InProceedings{pmlr-v100-evans20a, title = {Variational Optimization Based Reinforcement Learning for Infinite Dimensional Stochastic Systems}, author = {Evans, Ethan N. and Periera, Marcus A. and Boutselis, George I. and Theodorou, Evangelos A.}, booktitle = {Proceedings of the Conference on Robot Learning}, pages = {1231--1246}, year = {2020}, editor = {Kaelbling, Leslie Pack and Kragic, Danica and Sugiura, Komei}, volume = {100}, series = {Proceedings of Machine Learning Research}, month = {30 Oct--01 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v100/evans20a/evans20a.pdf}, url = {https://proceedings.mlr.press/v100/evans20a.html}, abstract = {Systems involving Partial Differential Equations (PDEs) have recently become more popular among the machine learning community. However prior methods usually treat infinite dimensional problems in finite dimensions with Reduced Order Models. This leads to committing to specific approximation schemes and subsequent derivation of control laws. Additionally, prior work does not consider spatio-temporal descriptions of noise that realistically represent the stochastic nature of physical systems. In this paper we suggest a new reinforcement learning framework that is mostly model-free for Stochastic PDEs with additive spacetime noise, based on variational optimization in infinite dimensions. In addition, our algorithm incorporates sparse representations that allow for efficient learning of feedback policies in high dimensions. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.} }
Endnote
%0 Conference Paper %T Variational Optimization Based Reinforcement Learning for Infinite Dimensional Stochastic Systems %A Ethan N. Evans %A Marcus A. Periera %A George I. Boutselis %A Evangelos A. Theodorou %B Proceedings of the Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2020 %E Leslie Pack Kaelbling %E Danica Kragic %E Komei Sugiura %F pmlr-v100-evans20a %I PMLR %P 1231--1246 %U https://proceedings.mlr.press/v100/evans20a.html %V 100 %X Systems involving Partial Differential Equations (PDEs) have recently become more popular among the machine learning community. However prior methods usually treat infinite dimensional problems in finite dimensions with Reduced Order Models. This leads to committing to specific approximation schemes and subsequent derivation of control laws. Additionally, prior work does not consider spatio-temporal descriptions of noise that realistically represent the stochastic nature of physical systems. In this paper we suggest a new reinforcement learning framework that is mostly model-free for Stochastic PDEs with additive spacetime noise, based on variational optimization in infinite dimensions. In addition, our algorithm incorporates sparse representations that allow for efficient learning of feedback policies in high dimensions. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.
APA
Evans, E.N., Periera, M.A., Boutselis, G.I. & Theodorou, E.A.. (2020). Variational Optimization Based Reinforcement Learning for Infinite Dimensional Stochastic Systems. Proceedings of the Conference on Robot Learning, in Proceedings of Machine Learning Research 100:1231-1246 Available from https://proceedings.mlr.press/v100/evans20a.html.

Related Material