Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation

Gregory Kahn, Adam Villaflor, Pieter Abbeel, Sergey Levine
Proceedings of The 2nd Conference on Robot Learning, PMLR 87:806-816, 2018.

Abstract

A general-purpose intelligent robot must be able to learn autonomously and be able to accomplish multiple tasks in order to be deployed in the real world. However, standard reinforcement learning approaches learn separate task-specific policies and assume the reward function for each task is known a priori. We propose a framework that learns event cues from off-policy data, and can flexibly combine these event cues at test time to accomplish different tasks. These event cue labels are not assumed to be known a priori, but are instead labeled using learned models, such as computer vision detectors, and then “backed up” in time using an action-conditioned predictive model. We show that a simulated robotic car and a real-world RC car can gather data and train fully autonomously without any human-provided labels beyond those needed to train the detectors, and then at test-time be able to accomplish a variety of different tasks. Videos of the experiments and code can be found at github.com/gkahn13/CAPs

Cite this Paper

BibTeX
@InProceedings{pmlr-v87-kahn18a, title = {Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation}, author = {Kahn, Gregory and Villaflor, Adam and Abbeel, Pieter and Levine, Sergey}, booktitle = {Proceedings of The 2nd Conference on Robot Learning}, pages = {806--816}, year = {2018}, editor = {Billard, Aude and Dragan, Anca and Peters, Jan and Morimoto, Jun}, volume = {87}, series = {Proceedings of Machine Learning Research}, month = {29--31 Oct}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v87/kahn18a/kahn18a.pdf}, url = {https://proceedings.mlr.press/v87/kahn18a.html}, abstract = {A general-purpose intelligent robot must be able to learn autonomously and be able to accomplish multiple tasks in order to be deployed in the real world. However, standard reinforcement learning approaches learn separate task-specific policies and assume the reward function for each task is known a priori. We propose a framework that learns event cues from off-policy data, and can flexibly combine these event cues at test time to accomplish different tasks. These event cue labels are not assumed to be known a priori, but are instead labeled using learned models, such as computer vision detectors, and then “backed up” in time using an action-conditioned predictive model. We show that a simulated robotic car and a real-world RC car can gather data and train fully autonomously without any human-provided labels beyond those needed to train the detectors, and then at test-time be able to accomplish a variety of different tasks. Videos of the experiments and code can be found at github.com/gkahn13/CAPs } }
Endnote
%0 Conference Paper %T Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation %A Gregory Kahn %A Adam Villaflor %A Pieter Abbeel %A Sergey Levine %B Proceedings of The 2nd Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2018 %E Aude Billard %E Anca Dragan %E Jan Peters %E Jun Morimoto %F pmlr-v87-kahn18a %I PMLR %P 806--816 %U https://proceedings.mlr.press/v87/kahn18a.html %V 87 %X A general-purpose intelligent robot must be able to learn autonomously and be able to accomplish multiple tasks in order to be deployed in the real world. However, standard reinforcement learning approaches learn separate task-specific policies and assume the reward function for each task is known a priori. We propose a framework that learns event cues from off-policy data, and can flexibly combine these event cues at test time to accomplish different tasks. These event cue labels are not assumed to be known a priori, but are instead labeled using learned models, such as computer vision detectors, and then “backed up” in time using an action-conditioned predictive model. We show that a simulated robotic car and a real-world RC car can gather data and train fully autonomously without any human-provided labels beyond those needed to train the detectors, and then at test-time be able to accomplish a variety of different tasks. Videos of the experiments and code can be found at github.com/gkahn13/CAPs
APA
Kahn, G., Villaflor, A., Abbeel, P. & Levine, S.. (2018). Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation. Proceedings of The 2nd Conference on Robot Learning, in Proceedings of Machine Learning Research 87:806-816 Available from https://proceedings.mlr.press/v87/kahn18a.html.

Related Material