Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Models

Jan Achterhold, Joerg Stueckler
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:3529-3537, 2021.

Abstract

In this paper, we learn dynamics models for parametrized families of dynamical systems with varying properties. The dynamics models are formulated as stochastic processes conditioned on a latent context variable which is inferred from observed transitions of the respective system. The probabilistic formulation allows us to compute an action sequence which, for a limited number of environment interactions, optimally explores the given system within the parametrized family. This is achieved by steering the system through transitions being most informative for the context variable. We demonstrate the effectiveness of our method for exploration on a non-linear toy-problem and two well-known reinforcement learning environments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-achterhold21a, title = { Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Models }, author = {Achterhold, Jan and Stueckler, Joerg}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {3529--3537}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/achterhold21a/achterhold21a.pdf}, url = {https://proceedings.mlr.press/v130/achterhold21a.html}, abstract = { In this paper, we learn dynamics models for parametrized families of dynamical systems with varying properties. The dynamics models are formulated as stochastic processes conditioned on a latent context variable which is inferred from observed transitions of the respective system. The probabilistic formulation allows us to compute an action sequence which, for a limited number of environment interactions, optimally explores the given system within the parametrized family. This is achieved by steering the system through transitions being most informative for the context variable. We demonstrate the effectiveness of our method for exploration on a non-linear toy-problem and two well-known reinforcement learning environments. } }
Endnote
%0 Conference Paper %T Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Models %A Jan Achterhold %A Joerg Stueckler %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-achterhold21a %I PMLR %P 3529--3537 %U https://proceedings.mlr.press/v130/achterhold21a.html %V 130 %X In this paper, we learn dynamics models for parametrized families of dynamical systems with varying properties. The dynamics models are formulated as stochastic processes conditioned on a latent context variable which is inferred from observed transitions of the respective system. The probabilistic formulation allows us to compute an action sequence which, for a limited number of environment interactions, optimally explores the given system within the parametrized family. This is achieved by steering the system through transitions being most informative for the context variable. We demonstrate the effectiveness of our method for exploration on a non-linear toy-problem and two well-known reinforcement learning environments.
APA
Achterhold, J. & Stueckler, J.. (2021). Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Models . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:3529-3537 Available from https://proceedings.mlr.press/v130/achterhold21a.html.

Related Material