Motion Perception in Reinforcement Learning with Dynamic Objects

Artemij Amiranashvili, Alexey Dosovitskiy, Vladlen Koltun, Thomas Brox
Proceedings of The 2nd Conference on Robot Learning, PMLR 87:156-168, 2018.

Abstract

In dynamic environments, learned controllers are supposed to take motion into account when selecting the action to be taken. However, in existing reinforcement learning works motion is rarely treated explicitly; it is rather assumed that the controller learns the necessary motion representation from temporal stacks of frames implicitly. In this paper, we show that for continuous control tasks learning an explicit representation of motion clearly improves the quality of the learned controller in dynamic scenarios. We demonstrate this on common benchmark tasks (Walker, Swimmer, Hopper), on target reaching and ball catching tasks with simulated robotic arms, and on a dynamic single ball juggling task. Moreover, we find that when equipped with an appropriate network architecture, the agent can, on some tasks, learn motion features also with pure reinforcement learning, without additional supervision.

Cite this Paper

BibTeX
@InProceedings{pmlr-v87-amiranashvili18a, title = {Motion Perception in Reinforcement Learning with Dynamic Objects}, author = {Amiranashvili, Artemij and Dosovitskiy, Alexey and Koltun, Vladlen and Brox, Thomas}, booktitle = {Proceedings of The 2nd Conference on Robot Learning}, pages = {156--168}, year = {2018}, editor = {Billard, Aude and Dragan, Anca and Peters, Jan and Morimoto, Jun}, volume = {87}, series = {Proceedings of Machine Learning Research}, month = {29--31 Oct}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v87/amiranashvili18a/amiranashvili18a.pdf}, url = {https://proceedings.mlr.press/v87/amiranashvili18a.html}, abstract = {In dynamic environments, learned controllers are supposed to take motion into account when selecting the action to be taken. However, in existing reinforcement learning works motion is rarely treated explicitly; it is rather assumed that the controller learns the necessary motion representation from temporal stacks of frames implicitly. In this paper, we show that for continuous control tasks learning an explicit representation of motion clearly improves the quality of the learned controller in dynamic scenarios. We demonstrate this on common benchmark tasks (Walker, Swimmer, Hopper), on target reaching and ball catching tasks with simulated robotic arms, and on a dynamic single ball juggling task. Moreover, we find that when equipped with an appropriate network architecture, the agent can, on some tasks, learn motion features also with pure reinforcement learning, without additional supervision. } }
Endnote
%0 Conference Paper %T Motion Perception in Reinforcement Learning with Dynamic Objects %A Artemij Amiranashvili %A Alexey Dosovitskiy %A Vladlen Koltun %A Thomas Brox %B Proceedings of The 2nd Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2018 %E Aude Billard %E Anca Dragan %E Jan Peters %E Jun Morimoto %F pmlr-v87-amiranashvili18a %I PMLR %P 156--168 %U https://proceedings.mlr.press/v87/amiranashvili18a.html %V 87 %X In dynamic environments, learned controllers are supposed to take motion into account when selecting the action to be taken. However, in existing reinforcement learning works motion is rarely treated explicitly; it is rather assumed that the controller learns the necessary motion representation from temporal stacks of frames implicitly. In this paper, we show that for continuous control tasks learning an explicit representation of motion clearly improves the quality of the learned controller in dynamic scenarios. We demonstrate this on common benchmark tasks (Walker, Swimmer, Hopper), on target reaching and ball catching tasks with simulated robotic arms, and on a dynamic single ball juggling task. Moreover, we find that when equipped with an appropriate network architecture, the agent can, on some tasks, learn motion features also with pure reinforcement learning, without additional supervision.
APA
Amiranashvili, A., Dosovitskiy, A., Koltun, V. & Brox, T.. (2018). Motion Perception in Reinforcement Learning with Dynamic Objects. Proceedings of The 2nd Conference on Robot Learning, in Proceedings of Machine Learning Research 87:156-168 Available from https://proceedings.mlr.press/v87/amiranashvili18a.html.

Related Material