Learning to Regrasp by Learning to Place

Shuo Cheng, Kaichun Mo, Lin Shao
Proceedings of the 5th Conference on Robot Learning, PMLR 164:277-286, 2022.

Abstract

In this paper, we explore whether a robot can learn to regrasp a diverse set of objects to achieve various desired grasp poses. Regrasping is needed whenever a robot’s current grasp pose fails to perform desired manipulation tasks. Endowing robots with such an ability has applications in many domains such as manufacturing or domestic services. Yet, it is a challenging task due to the large diversity of geometry in everyday objects and the high dimensionality of the state and action space. In this paper, we propose a system for robots to take partial point clouds of an object and the supporting environment as inputs and output a sequence of pick-and-place operations to transform an initial object grasp pose to the desired object grasp poses. The key technique includes a neural stable placement predictor and a regrasp graph based solution through leveraging and changing the surrounding environment. We introduce a new and challenging synthetic dataset for learning and evaluating the proposed approach. We demonstrate the effectiveness of our proposed system with both simulator and real-world experiments. More videos and visualization examples are available on our project https://sites.google.com/view/regrasp.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-cheng22a, title = {Learning to Regrasp by Learning to Place}, author = {Cheng, Shuo and Mo, Kaichun and Shao, Lin}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {277--286}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/cheng22a/cheng22a.pdf}, url = {https://proceedings.mlr.press/v164/cheng22a.html}, abstract = {In this paper, we explore whether a robot can learn to regrasp a diverse set of objects to achieve various desired grasp poses. Regrasping is needed whenever a robot’s current grasp pose fails to perform desired manipulation tasks. Endowing robots with such an ability has applications in many domains such as manufacturing or domestic services. Yet, it is a challenging task due to the large diversity of geometry in everyday objects and the high dimensionality of the state and action space. In this paper, we propose a system for robots to take partial point clouds of an object and the supporting environment as inputs and output a sequence of pick-and-place operations to transform an initial object grasp pose to the desired object grasp poses. The key technique includes a neural stable placement predictor and a regrasp graph based solution through leveraging and changing the surrounding environment. We introduce a new and challenging synthetic dataset for learning and evaluating the proposed approach. We demonstrate the effectiveness of our proposed system with both simulator and real-world experiments. More videos and visualization examples are available on our project https://sites.google.com/view/regrasp.} }
Endnote
%0 Conference Paper %T Learning to Regrasp by Learning to Place %A Shuo Cheng %A Kaichun Mo %A Lin Shao %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-cheng22a %I PMLR %P 277--286 %U https://proceedings.mlr.press/v164/cheng22a.html %V 164 %X In this paper, we explore whether a robot can learn to regrasp a diverse set of objects to achieve various desired grasp poses. Regrasping is needed whenever a robot’s current grasp pose fails to perform desired manipulation tasks. Endowing robots with such an ability has applications in many domains such as manufacturing or domestic services. Yet, it is a challenging task due to the large diversity of geometry in everyday objects and the high dimensionality of the state and action space. In this paper, we propose a system for robots to take partial point clouds of an object and the supporting environment as inputs and output a sequence of pick-and-place operations to transform an initial object grasp pose to the desired object grasp poses. The key technique includes a neural stable placement predictor and a regrasp graph based solution through leveraging and changing the surrounding environment. We introduce a new and challenging synthetic dataset for learning and evaluating the proposed approach. We demonstrate the effectiveness of our proposed system with both simulator and real-world experiments. More videos and visualization examples are available on our project https://sites.google.com/view/regrasp.
APA
Cheng, S., Mo, K. & Shao, L.. (2022). Learning to Regrasp by Learning to Place. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:277-286 Available from https://proceedings.mlr.press/v164/cheng22a.html.

Related Material