Self-Supervised Learning of Long-Horizon Manipulation Tasks with Finite-State Task Machines

Junchi Liang, Abdeslam Boularias
Proceedings of the 3rd Conference on Learning for Dynamics and Control, PMLR 144:484-497, 2021.

Abstract

We consider the problem of a robot learning to manipulate unknown objects while using them to perform a complex task that is composed of several sub-tasks. The robot receives 6D poses of the objects along with their semantic labels, and executes nonprehensile actions on them. The robot does not receive any feedback regarding the task until the end of an episode, where a binary reward indicates success or failure in performing the task. Moreover, certain attributes of objects cannot be always observed, so the robot needs to learn to remember pertinent past actions that it executed. We propose to solve this problem by simultaneously learning a low-level control policy and a high-level finite-state task machine that keeps track of the progress made by the robot in solving the various sub-tasks and guides the low-level policy. Several experiments in simulation clearly show that the proposed approach is efficient at solving complex robotic tasks without any supervision.

Cite this Paper

BibTeX
@InProceedings{pmlr-v144-liang21a, title = {Self-Supervised Learning of Long-Horizon Manipulation Tasks with Finite-State Task Machines}, author = {Liang, Junchi and Boularias, Abdeslam}, booktitle = {Proceedings of the 3rd Conference on Learning for Dynamics and Control}, pages = {484--497}, year = {2021}, editor = {Jadbabaie, Ali and Lygeros, John and Pappas, George J. and A. Parrilo, Pablo and Recht, Benjamin and Tomlin, Claire J. and Zeilinger, Melanie N.}, volume = {144}, series = {Proceedings of Machine Learning Research}, month = {07 -- 08 June}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v144/liang21a/liang21a.pdf}, url = {https://proceedings.mlr.press/v144/liang21a.html}, abstract = {We consider the problem of a robot learning to manipulate unknown objects while using them to perform a complex task that is composed of several sub-tasks. The robot receives 6D poses of the objects along with their semantic labels, and executes nonprehensile actions on them. The robot does not receive any feedback regarding the task until the end of an episode, where a binary reward indicates success or failure in performing the task. Moreover, certain attributes of objects cannot be always observed, so the robot needs to learn to remember pertinent past actions that it executed. We propose to solve this problem by simultaneously learning a low-level control policy and a high-level finite-state task machine that keeps track of the progress made by the robot in solving the various sub-tasks and guides the low-level policy. Several experiments in simulation clearly show that the proposed approach is efficient at solving complex robotic tasks without any supervision.} }
Endnote
%0 Conference Paper %T Self-Supervised Learning of Long-Horizon Manipulation Tasks with Finite-State Task Machines %A Junchi Liang %A Abdeslam Boularias %B Proceedings of the 3rd Conference on Learning for Dynamics and Control %C Proceedings of Machine Learning Research %D 2021 %E Ali Jadbabaie %E John Lygeros %E George J. Pappas %E Pablo A. Parrilo %E Benjamin Recht %E Claire J. Tomlin %E Melanie N. Zeilinger %F pmlr-v144-liang21a %I PMLR %P 484--497 %U https://proceedings.mlr.press/v144/liang21a.html %V 144 %X We consider the problem of a robot learning to manipulate unknown objects while using them to perform a complex task that is composed of several sub-tasks. The robot receives 6D poses of the objects along with their semantic labels, and executes nonprehensile actions on them. The robot does not receive any feedback regarding the task until the end of an episode, where a binary reward indicates success or failure in performing the task. Moreover, certain attributes of objects cannot be always observed, so the robot needs to learn to remember pertinent past actions that it executed. We propose to solve this problem by simultaneously learning a low-level control policy and a high-level finite-state task machine that keeps track of the progress made by the robot in solving the various sub-tasks and guides the low-level policy. Several experiments in simulation clearly show that the proposed approach is efficient at solving complex robotic tasks without any supervision.
APA
Liang, J. & Boularias, A.. (2021). Self-Supervised Learning of Long-Horizon Manipulation Tasks with Finite-State Task Machines. Proceedings of the 3rd Conference on Learning for Dynamics and Control, in Proceedings of Machine Learning Research 144:484-497 Available from https://proceedings.mlr.press/v144/liang21a.html.

Related Material