CLOUD: Contrastive Learning of Unsupervised Dynamics

Jianren Wang, Yujie Lu, Hang Zhao
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:365-376, 2021.

Abstract

Developing agents that can perform complex control tasks from high dimensional observations such as pixels is challenging due to difficulties in learning dynamics efficiently. In this work, we propose to learn forward and inverse dynamics in a fully unsupervised manner via contrastive estimation. Specifically, we train a forward dynamics model and an inverse dynamics model in the feature space of states and actions with data collected from random exploration. Unlike most existing deterministic models, our energy-based model takes into account the stochastic nature of agent-environment interactions. We demonstrate the efficacy of our approach across a variety of tasks including goal-directed planning and imitation from observations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-wang21c, title = {CLOUD: Contrastive Learning of Unsupervised Dynamics}, author = {Wang, Jianren and Lu, Yujie and Zhao, Hang}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {365--376}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/wang21c/wang21c.pdf}, url = {https://proceedings.mlr.press/v155/wang21c.html}, abstract = {Developing agents that can perform complex control tasks from high dimensional observations such as pixels is challenging due to difficulties in learning dynamics efficiently. In this work, we propose to learn forward and inverse dynamics in a fully unsupervised manner via contrastive estimation. Specifically, we train a forward dynamics model and an inverse dynamics model in the feature space of states and actions with data collected from random exploration. Unlike most existing deterministic models, our energy-based model takes into account the stochastic nature of agent-environment interactions. We demonstrate the efficacy of our approach across a variety of tasks including goal-directed planning and imitation from observations.} }
Endnote
%0 Conference Paper %T CLOUD: Contrastive Learning of Unsupervised Dynamics %A Jianren Wang %A Yujie Lu %A Hang Zhao %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-wang21c %I PMLR %P 365--376 %U https://proceedings.mlr.press/v155/wang21c.html %V 155 %X Developing agents that can perform complex control tasks from high dimensional observations such as pixels is challenging due to difficulties in learning dynamics efficiently. In this work, we propose to learn forward and inverse dynamics in a fully unsupervised manner via contrastive estimation. Specifically, we train a forward dynamics model and an inverse dynamics model in the feature space of states and actions with data collected from random exploration. Unlike most existing deterministic models, our energy-based model takes into account the stochastic nature of agent-environment interactions. We demonstrate the efficacy of our approach across a variety of tasks including goal-directed planning and imitation from observations.
APA
Wang, J., Lu, Y. & Zhao, H.. (2021). CLOUD: Contrastive Learning of Unsupervised Dynamics. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:365-376 Available from https://proceedings.mlr.press/v155/wang21c.html.

Related Material