PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards

Prasoon Goyal, Scott Niekum, Raymond Mooney
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:485-497, 2021.

Abstract

Reinforcement learning (RL), particularly in sparse reward settings, often requires prohibitively large numbers of interactions with the environment, thereby limiting its applicability to complex problems. To address this, several prior approaches have used natural language to guide the agent’s exploration. However, these approaches typically operate on structured representations of the environment, and/or assume some structure in the natural language commands. In this work, we propose a model that directly maps pixels to rewards, given a free-form natural language description of the task, which can then be used for policy learning. Our experiments on the Meta-World robot manipulation domain show that language-based rewards significantly improves the sample efficiency of policy learning, both in sparse and dense reward settings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-goyal21a, title = {PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards}, author = {Goyal, Prasoon and Niekum, Scott and Mooney, Raymond}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {485--497}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/goyal21a/goyal21a.pdf}, url = {https://proceedings.mlr.press/v155/goyal21a.html}, abstract = {Reinforcement learning (RL), particularly in sparse reward settings, often requires prohibitively large numbers of interactions with the environment, thereby limiting its applicability to complex problems. To address this, several prior approaches have used natural language to guide the agent’s exploration. However, these approaches typically operate on structured representations of the environment, and/or assume some structure in the natural language commands. In this work, we propose a model that directly maps pixels to rewards, given a free-form natural language description of the task, which can then be used for policy learning. Our experiments on the Meta-World robot manipulation domain show that language-based rewards significantly improves the sample efficiency of policy learning, both in sparse and dense reward settings.} }
Endnote
%0 Conference Paper %T PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards %A Prasoon Goyal %A Scott Niekum %A Raymond Mooney %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-goyal21a %I PMLR %P 485--497 %U https://proceedings.mlr.press/v155/goyal21a.html %V 155 %X Reinforcement learning (RL), particularly in sparse reward settings, often requires prohibitively large numbers of interactions with the environment, thereby limiting its applicability to complex problems. To address this, several prior approaches have used natural language to guide the agent’s exploration. However, these approaches typically operate on structured representations of the environment, and/or assume some structure in the natural language commands. In this work, we propose a model that directly maps pixels to rewards, given a free-form natural language description of the task, which can then be used for policy learning. Our experiments on the Meta-World robot manipulation domain show that language-based rewards significantly improves the sample efficiency of policy learning, both in sparse and dense reward settings.
APA
Goyal, P., Niekum, S. & Mooney, R.. (2021). PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:485-497 Available from https://proceedings.mlr.press/v155/goyal21a.html.

Related Material