Efficient Tactile Simulation with Differentiability for Robotic Manipulation

Jie Xu, Sangwoon Kim, Tao Chen, Alberto Rodriguez Garcia, Pulkit Agrawal, Wojciech Matusik, Shinjiro Sueda
Proceedings of The 6th Conference on Robot Learning, PMLR 205:1488-1498, 2023.

Abstract

Efficient simulation of tactile sensors can unlock new opportunities for learning tactile-based manipulation policies in simulation and then transferring the learned policy to real systems, but fast and reliable simulators for dense tactile normal and shear force fields are still under-explored. We present a novel approach for efficiently simulating both the normal and shear tactile force field covering the entire contact surface with an arbitrary tactile sensor spatial layout. Our simulator also provides analytical gradients of the tactile forces to accelerate policy learning. We conduct extensive simulation experiments to showcase our approach and demonstrate successful zero-shot sim-to-real transfer for a high-precision peg-insertion task with high-resolution vision-based GelSlim tactile sensors.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-xu23b, title = {Efficient Tactile Simulation with Differentiability for Robotic Manipulation}, author = {Xu, Jie and Kim, Sangwoon and Chen, Tao and Garcia, Alberto Rodriguez and Agrawal, Pulkit and Matusik, Wojciech and Sueda, Shinjiro}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {1488--1498}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/xu23b/xu23b.pdf}, url = {https://proceedings.mlr.press/v205/xu23b.html}, abstract = {Efficient simulation of tactile sensors can unlock new opportunities for learning tactile-based manipulation policies in simulation and then transferring the learned policy to real systems, but fast and reliable simulators for dense tactile normal and shear force fields are still under-explored. We present a novel approach for efficiently simulating both the normal and shear tactile force field covering the entire contact surface with an arbitrary tactile sensor spatial layout. Our simulator also provides analytical gradients of the tactile forces to accelerate policy learning. We conduct extensive simulation experiments to showcase our approach and demonstrate successful zero-shot sim-to-real transfer for a high-precision peg-insertion task with high-resolution vision-based GelSlim tactile sensors.} }
Endnote
%0 Conference Paper %T Efficient Tactile Simulation with Differentiability for Robotic Manipulation %A Jie Xu %A Sangwoon Kim %A Tao Chen %A Alberto Rodriguez Garcia %A Pulkit Agrawal %A Wojciech Matusik %A Shinjiro Sueda %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-xu23b %I PMLR %P 1488--1498 %U https://proceedings.mlr.press/v205/xu23b.html %V 205 %X Efficient simulation of tactile sensors can unlock new opportunities for learning tactile-based manipulation policies in simulation and then transferring the learned policy to real systems, but fast and reliable simulators for dense tactile normal and shear force fields are still under-explored. We present a novel approach for efficiently simulating both the normal and shear tactile force field covering the entire contact surface with an arbitrary tactile sensor spatial layout. Our simulator also provides analytical gradients of the tactile forces to accelerate policy learning. We conduct extensive simulation experiments to showcase our approach and demonstrate successful zero-shot sim-to-real transfer for a high-precision peg-insertion task with high-resolution vision-based GelSlim tactile sensors.
APA
Xu, J., Kim, S., Chen, T., Garcia, A.R., Agrawal, P., Matusik, W. & Sueda, S.. (2023). Efficient Tactile Simulation with Differentiability for Robotic Manipulation. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:1488-1498 Available from https://proceedings.mlr.press/v205/xu23b.html.

Related Material