NeuralGrasps: Learning Implicit Representations for Grasps of Multiple Robotic Hands

Ninad Khargonkar, Neil Song, Zesheng Xu, B Prabhakaran, Yu Xiang
Proceedings of The 6th Conference on Robot Learning, PMLR 205:516-526, 2023.

Abstract

We introduce a neural implicit representation for grasps of objects from multiple robotic hands. Different grasps across multiple robotic hands are encoded into a shared latent space. Each latent vector is learned to decode to the 3D shape of an object and the 3D shape of a robotic hand in a grasping pose in terms of the signed distance functions of the two 3D shapes. In addition, the distance metric in the latent space is learned to preserve the similarity between grasps across different robotic hands, where the similarity of grasps is defined according to contact regions of the robotic hands. This property enables our method to transfer grasps between different grippers including a human hand, and grasp transfer has the potential to share grasping skills between robots and enable robots to learn grasping skills from humans. Furthermore, the encoded signed distance functions of objects and grasps in our implicit representation can be used for 6D object pose estimation with grasping contact optimization from partial point clouds, which enables robotic grasping in the real world.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-khargonkar23a, title = {NeuralGrasps: Learning Implicit Representations for Grasps of Multiple Robotic Hands}, author = {Khargonkar, Ninad and Song, Neil and Xu, Zesheng and Prabhakaran, B and Xiang, Yu}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {516--526}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/khargonkar23a/khargonkar23a.pdf}, url = {https://proceedings.mlr.press/v205/khargonkar23a.html}, abstract = {We introduce a neural implicit representation for grasps of objects from multiple robotic hands. Different grasps across multiple robotic hands are encoded into a shared latent space. Each latent vector is learned to decode to the 3D shape of an object and the 3D shape of a robotic hand in a grasping pose in terms of the signed distance functions of the two 3D shapes. In addition, the distance metric in the latent space is learned to preserve the similarity between grasps across different robotic hands, where the similarity of grasps is defined according to contact regions of the robotic hands. This property enables our method to transfer grasps between different grippers including a human hand, and grasp transfer has the potential to share grasping skills between robots and enable robots to learn grasping skills from humans. Furthermore, the encoded signed distance functions of objects and grasps in our implicit representation can be used for 6D object pose estimation with grasping contact optimization from partial point clouds, which enables robotic grasping in the real world.} }
Endnote
%0 Conference Paper %T NeuralGrasps: Learning Implicit Representations for Grasps of Multiple Robotic Hands %A Ninad Khargonkar %A Neil Song %A Zesheng Xu %A B Prabhakaran %A Yu Xiang %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-khargonkar23a %I PMLR %P 516--526 %U https://proceedings.mlr.press/v205/khargonkar23a.html %V 205 %X We introduce a neural implicit representation for grasps of objects from multiple robotic hands. Different grasps across multiple robotic hands are encoded into a shared latent space. Each latent vector is learned to decode to the 3D shape of an object and the 3D shape of a robotic hand in a grasping pose in terms of the signed distance functions of the two 3D shapes. In addition, the distance metric in the latent space is learned to preserve the similarity between grasps across different robotic hands, where the similarity of grasps is defined according to contact regions of the robotic hands. This property enables our method to transfer grasps between different grippers including a human hand, and grasp transfer has the potential to share grasping skills between robots and enable robots to learn grasping skills from humans. Furthermore, the encoded signed distance functions of objects and grasps in our implicit representation can be used for 6D object pose estimation with grasping contact optimization from partial point clouds, which enables robotic grasping in the real world.
APA
Khargonkar, N., Song, N., Xu, Z., Prabhakaran, B. & Xiang, Y.. (2023). NeuralGrasps: Learning Implicit Representations for Grasps of Multiple Robotic Hands. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:516-526 Available from https://proceedings.mlr.press/v205/khargonkar23a.html.

Related Material