Localized active learning of Gaussian process state space models

Alexandre Capone, Gerrit Noske, Jonas Umlauft, Thomas Beckers, Armin Lederer, Sandra Hirche
Proceedings of the 2nd Conference on Learning for Dynamics and Control, PMLR 120:490-499, 2020.

Abstract

In learning based methods for dynamical systems, exploration plays a crucial role, as accurate models of the dynamics need to be learned. Most of the tools developed so far focus on a proper exploration-exploitation trade-off to solve the given task, or actively strive for unknown areas of the task space. However, in the latter case, the exploration is performed greedily, and fails to capture the effect that learning in the near future will have on model uncertainty in the distant future, effectively steering the system towards exploratory trajectories that yield little information. In this paper, we provide an information theory-based model predictive control method that anticipates the learning effect when exploring dynamical systems, and steers the system towards the most informative points. We employ a Gaussian process to model the system dynamics, which enables us to quantify the model uncertainty and estimate future information gains. We include a numerical example illustrates that illustrates the effectiveness of the proposed approach.

Cite this Paper

BibTeX
@InProceedings{pmlr-v120-capone20a, title = {Localized active learning of Gaussian process state space models}, author = {Capone, Alexandre and Noske, Gerrit and Umlauft, Jonas and Beckers, Thomas and Lederer, Armin and Hirche, Sandra}, booktitle = {Proceedings of the 2nd Conference on Learning for Dynamics and Control}, pages = {490--499}, year = {2020}, editor = {Bayen, Alexandre M. and Jadbabaie, Ali and Pappas, George and Parrilo, Pablo A. and Recht, Benjamin and Tomlin, Claire and Zeilinger, Melanie}, volume = {120}, series = {Proceedings of Machine Learning Research}, month = {10--11 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v120/capone20a/capone20a.pdf}, url = {https://proceedings.mlr.press/v120/capone20a.html}, abstract = {In learning based methods for dynamical systems, exploration plays a crucial role, as accurate models of the dynamics need to be learned. Most of the tools developed so far focus on a proper exploration-exploitation trade-off to solve the given task, or actively strive for unknown areas of the task space. However, in the latter case, the exploration is performed greedily, and fails to capture the effect that learning in the near future will have on model uncertainty in the distant future, effectively steering the system towards exploratory trajectories that yield little information. In this paper, we provide an information theory-based model predictive control method that anticipates the learning effect when exploring dynamical systems, and steers the system towards the most informative points. We employ a Gaussian process to model the system dynamics, which enables us to quantify the model uncertainty and estimate future information gains. We include a numerical example illustrates that illustrates the effectiveness of the proposed approach.} }
Endnote
%0 Conference Paper %T Localized active learning of Gaussian process state space models %A Alexandre Capone %A Gerrit Noske %A Jonas Umlauft %A Thomas Beckers %A Armin Lederer %A Sandra Hirche %B Proceedings of the 2nd Conference on Learning for Dynamics and Control %C Proceedings of Machine Learning Research %D 2020 %E Alexandre M. Bayen %E Ali Jadbabaie %E George Pappas %E Pablo A. Parrilo %E Benjamin Recht %E Claire Tomlin %E Melanie Zeilinger %F pmlr-v120-capone20a %I PMLR %P 490--499 %U https://proceedings.mlr.press/v120/capone20a.html %V 120 %X In learning based methods for dynamical systems, exploration plays a crucial role, as accurate models of the dynamics need to be learned. Most of the tools developed so far focus on a proper exploration-exploitation trade-off to solve the given task, or actively strive for unknown areas of the task space. However, in the latter case, the exploration is performed greedily, and fails to capture the effect that learning in the near future will have on model uncertainty in the distant future, effectively steering the system towards exploratory trajectories that yield little information. In this paper, we provide an information theory-based model predictive control method that anticipates the learning effect when exploring dynamical systems, and steers the system towards the most informative points. We employ a Gaussian process to model the system dynamics, which enables us to quantify the model uncertainty and estimate future information gains. We include a numerical example illustrates that illustrates the effectiveness of the proposed approach.
APA
Capone, A., Noske, G., Umlauft, J., Beckers, T., Lederer, A. & Hirche, S.. (2020). Localized active learning of Gaussian process state space models. Proceedings of the 2nd Conference on Learning for Dynamics and Control, in Proceedings of Machine Learning Research 120:490-499 Available from https://proceedings.mlr.press/v120/capone20a.html.

Related Material