ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning

Yufei Wang, Narasimhan Gautham, Xingyu Lin, Brian Okorn, David Held
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:1030-1048, 2021.

Abstract

Current image-based reinforcement learning (RL) algorithms typically operate on the whole image without performing object-level reasoning. This leads to inefficient goal sampling and ineffective reward functions. In this paper, we improve upon previous visual self-supervised RL by incorporating object-level reasoning and occlusion reasoning. Specifically, we use unknown object segmentation to ignore distractors in the scene for better reward computation and goal generation; we further enable occlusion reasoning by employing a novel auxiliary loss and training scheme. We demonstrate that our proposed algorithm, ROLL (Reinforcement learning with Object Level Learning), learns dramatically faster and achieves better final performance compared with previous methods in several simulated visual control tasks. Project video and code are available at https://sites.google.com/andrew.cmu.edu/roll.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-wang21e, title = {ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning}, author = {Wang, Yufei and Gautham, Narasimhan and Lin, Xingyu and Okorn, Brian and Held, David}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {1030--1048}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/wang21e/wang21e.pdf}, url = {https://proceedings.mlr.press/v155/wang21e.html}, abstract = {Current image-based reinforcement learning (RL) algorithms typically operate on the whole image without performing object-level reasoning. This leads to inefficient goal sampling and ineffective reward functions. In this paper, we improve upon previous visual self-supervised RL by incorporating object-level reasoning and occlusion reasoning. Specifically, we use unknown object segmentation to ignore distractors in the scene for better reward computation and goal generation; we further enable occlusion reasoning by employing a novel auxiliary loss and training scheme. We demonstrate that our proposed algorithm, ROLL (Reinforcement learning with Object Level Learning), learns dramatically faster and achieves better final performance compared with previous methods in several simulated visual control tasks. Project video and code are available at https://sites.google.com/andrew.cmu.edu/roll.} }
Endnote
%0 Conference Paper %T ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning %A Yufei Wang %A Narasimhan Gautham %A Xingyu Lin %A Brian Okorn %A David Held %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-wang21e %I PMLR %P 1030--1048 %U https://proceedings.mlr.press/v155/wang21e.html %V 155 %X Current image-based reinforcement learning (RL) algorithms typically operate on the whole image without performing object-level reasoning. This leads to inefficient goal sampling and ineffective reward functions. In this paper, we improve upon previous visual self-supervised RL by incorporating object-level reasoning and occlusion reasoning. Specifically, we use unknown object segmentation to ignore distractors in the scene for better reward computation and goal generation; we further enable occlusion reasoning by employing a novel auxiliary loss and training scheme. We demonstrate that our proposed algorithm, ROLL (Reinforcement learning with Object Level Learning), learns dramatically faster and achieves better final performance compared with previous methods in several simulated visual control tasks. Project video and code are available at https://sites.google.com/andrew.cmu.edu/roll.
APA
Wang, Y., Gautham, N., Lin, X., Okorn, B. & Held, D.. (2021). ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:1030-1048 Available from https://proceedings.mlr.press/v155/wang21e.html.

Related Material