Learning 6-DoF Grasping and Pick-Place Using Attention Focus

Marcus Gualtieri, Robert Platt
Proceedings of The 2nd Conference on Robot Learning, PMLR 87:477-486, 2018.

Abstract

We address a class of manipulation problems where the robot perceives the scene with a depth sensor and can move its end effector in a space with six degrees of freedom—3D position and orientation. Our approach is to formulate the problem as a Markov decision process (MDP) with abstract yet generally applicable state and action representations. Finding a good solution to the MDP requires adding constraints on the allowed actions. We develop a specific set of constraints called hierarchical SE(3) sampling (HSE3S) which causes the robot to learn a sequence of gazes to focus attention on the task-relevant parts of the scene. We demonstrate the effectiveness of our approach on three challenging pick-place tasks (with novel objects in clutter and nontrivial places) both in simulation and on a real robot, even though all training is done in simulation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v87-gualtieri18a, title = {Learning 6-DoF Grasping and Pick-Place Using Attention Focus}, author = {Gualtieri, Marcus and Platt, Robert}, booktitle = {Proceedings of The 2nd Conference on Robot Learning}, pages = {477--486}, year = {2018}, editor = {Billard, Aude and Dragan, Anca and Peters, Jan and Morimoto, Jun}, volume = {87}, series = {Proceedings of Machine Learning Research}, month = {29--31 Oct}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v87/gualtieri18a/gualtieri18a.pdf}, url = {https://proceedings.mlr.press/v87/gualtieri18a.html}, abstract = {We address a class of manipulation problems where the robot perceives the scene with a depth sensor and can move its end effector in a space with six degrees of freedom—3D position and orientation. Our approach is to formulate the problem as a Markov decision process (MDP) with abstract yet generally applicable state and action representations. Finding a good solution to the MDP requires adding constraints on the allowed actions. We develop a specific set of constraints called hierarchical SE(3) sampling (HSE3S) which causes the robot to learn a sequence of gazes to focus attention on the task-relevant parts of the scene. We demonstrate the effectiveness of our approach on three challenging pick-place tasks (with novel objects in clutter and nontrivial places) both in simulation and on a real robot, even though all training is done in simulation. } }
Endnote
%0 Conference Paper %T Learning 6-DoF Grasping and Pick-Place Using Attention Focus %A Marcus Gualtieri %A Robert Platt %B Proceedings of The 2nd Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2018 %E Aude Billard %E Anca Dragan %E Jan Peters %E Jun Morimoto %F pmlr-v87-gualtieri18a %I PMLR %P 477--486 %U https://proceedings.mlr.press/v87/gualtieri18a.html %V 87 %X We address a class of manipulation problems where the robot perceives the scene with a depth sensor and can move its end effector in a space with six degrees of freedom—3D position and orientation. Our approach is to formulate the problem as a Markov decision process (MDP) with abstract yet generally applicable state and action representations. Finding a good solution to the MDP requires adding constraints on the allowed actions. We develop a specific set of constraints called hierarchical SE(3) sampling (HSE3S) which causes the robot to learn a sequence of gazes to focus attention on the task-relevant parts of the scene. We demonstrate the effectiveness of our approach on three challenging pick-place tasks (with novel objects in clutter and nontrivial places) both in simulation and on a real robot, even though all training is done in simulation.
APA
Gualtieri, M. & Platt, R.. (2018). Learning 6-DoF Grasping and Pick-Place Using Attention Focus. Proceedings of The 2nd Conference on Robot Learning, in Proceedings of Machine Learning Research 87:477-486 Available from https://proceedings.mlr.press/v87/gualtieri18a.html.

Related Material