Entity Abstraction in Visual Model-Based Reinforcement Learning

Rishi Veerapaneni, John D. Co-Reyes, Michael Chang, Michael Janner, Chelsea Finn, Jiajun Wu, Joshua Tenenbaum, Sergey Levine
Proceedings of the Conference on Robot Learning, PMLR 100:1439-1456, 2020.

Abstract

We present OP3, a framework for model-based reinforcement learning that acquires object representations from raw visual observations without supervision and uses them to predict and plan. To ground these abstract representations of entities to actual objects in the world, we formulate an interactive inference algorithm which incorporates dynamic information in the scene. Our model can handle a variable number of entities by symmetrically processing each object representation with the same locally-scoped function. On block-stacking tasks, OP3 can generalize to novel block configurations and more objects than seen during training, outperforming both a model that assumes access to object supervision and a state-of-the-art video prediction model.

Cite this Paper

BibTeX
@InProceedings{pmlr-v100-veerapaneni20a, title = {Entity Abstraction in Visual Model-Based Reinforcement Learning}, author = {Veerapaneni, Rishi and Co-Reyes, John D. and Chang, Michael and Janner, Michael and Finn, Chelsea and Wu, Jiajun and Tenenbaum, Joshua and Levine, Sergey}, booktitle = {Proceedings of the Conference on Robot Learning}, pages = {1439--1456}, year = {2020}, editor = {Kaelbling, Leslie Pack and Kragic, Danica and Sugiura, Komei}, volume = {100}, series = {Proceedings of Machine Learning Research}, month = {30 Oct--01 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v100/veerapaneni20a/veerapaneni20a.pdf}, url = {https://proceedings.mlr.press/v100/veerapaneni20a.html}, abstract = {We present OP3, a framework for model-based reinforcement learning that acquires object representations from raw visual observations without supervision and uses them to predict and plan. To ground these abstract representations of entities to actual objects in the world, we formulate an interactive inference algorithm which incorporates dynamic information in the scene. Our model can handle a variable number of entities by symmetrically processing each object representation with the same locally-scoped function. On block-stacking tasks, OP3 can generalize to novel block configurations and more objects than seen during training, outperforming both a model that assumes access to object supervision and a state-of-the-art video prediction model.} }
Endnote
%0 Conference Paper %T Entity Abstraction in Visual Model-Based Reinforcement Learning %A Rishi Veerapaneni %A John D. Co-Reyes %A Michael Chang %A Michael Janner %A Chelsea Finn %A Jiajun Wu %A Joshua Tenenbaum %A Sergey Levine %B Proceedings of the Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2020 %E Leslie Pack Kaelbling %E Danica Kragic %E Komei Sugiura %F pmlr-v100-veerapaneni20a %I PMLR %P 1439--1456 %U https://proceedings.mlr.press/v100/veerapaneni20a.html %V 100 %X We present OP3, a framework for model-based reinforcement learning that acquires object representations from raw visual observations without supervision and uses them to predict and plan. To ground these abstract representations of entities to actual objects in the world, we formulate an interactive inference algorithm which incorporates dynamic information in the scene. Our model can handle a variable number of entities by symmetrically processing each object representation with the same locally-scoped function. On block-stacking tasks, OP3 can generalize to novel block configurations and more objects than seen during training, outperforming both a model that assumes access to object supervision and a state-of-the-art video prediction model.
APA
Veerapaneni, R., Co-Reyes, J.D., Chang, M., Janner, M., Finn, C., Wu, J., Tenenbaum, J. & Levine, S.. (2020). Entity Abstraction in Visual Model-Based Reinforcement Learning. Proceedings of the Conference on Robot Learning, in Proceedings of Machine Learning Research 100:1439-1456 Available from https://proceedings.mlr.press/v100/veerapaneni20a.html.

Related Material