Manipulation via Membranes: High-Resolution and Highly Deformable Tactile Sensing and Control

Miquel Oller, Mireia Planas i Lisbona, Dmitry Berenson, Nima Fazeli
Proceedings of The 6th Conference on Robot Learning, PMLR 205:1850-1859, 2023.

Abstract

Collocated tactile sensing is a fundamental enabling technology for dexterous manipulation. However, deformable sensors introduce complex dynamics between the robot, grasped object, and environment that must be considered for fine manipulation. Here, we propose a method to learn soft tactile sensor membrane dynamics that accounts for sensor deformations caused by the physical interaction between the grasped object and environment. Our method combines the perceived 3D geometry of the membrane with proprioceptive reaction wrenches to predict future deformations conditioned on robot action. Grasped object poses are recovered from membrane geometry and reaction wrenches, decoupling interaction dynamics from the tactile observation model. We benchmark our approach on two real-world contact-rich tasks: drawing with a grasped marker and in-hand pivoting. Our results suggest that explicitly modeling membrane dynamics achieves better task performance and generalization to unseen objects than baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-oller23a, title = {Manipulation via Membranes: High-Resolution and Highly Deformable Tactile Sensing and Control}, author = {Oller, Miquel and Lisbona, Mireia Planas i and Berenson, Dmitry and Fazeli, Nima}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {1850--1859}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/oller23a/oller23a.pdf}, url = {https://proceedings.mlr.press/v205/oller23a.html}, abstract = {Collocated tactile sensing is a fundamental enabling technology for dexterous manipulation. However, deformable sensors introduce complex dynamics between the robot, grasped object, and environment that must be considered for fine manipulation. Here, we propose a method to learn soft tactile sensor membrane dynamics that accounts for sensor deformations caused by the physical interaction between the grasped object and environment. Our method combines the perceived 3D geometry of the membrane with proprioceptive reaction wrenches to predict future deformations conditioned on robot action. Grasped object poses are recovered from membrane geometry and reaction wrenches, decoupling interaction dynamics from the tactile observation model. We benchmark our approach on two real-world contact-rich tasks: drawing with a grasped marker and in-hand pivoting. Our results suggest that explicitly modeling membrane dynamics achieves better task performance and generalization to unseen objects than baselines.} }
Endnote
%0 Conference Paper %T Manipulation via Membranes: High-Resolution and Highly Deformable Tactile Sensing and Control %A Miquel Oller %A Mireia Planas i Lisbona %A Dmitry Berenson %A Nima Fazeli %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-oller23a %I PMLR %P 1850--1859 %U https://proceedings.mlr.press/v205/oller23a.html %V 205 %X Collocated tactile sensing is a fundamental enabling technology for dexterous manipulation. However, deformable sensors introduce complex dynamics between the robot, grasped object, and environment that must be considered for fine manipulation. Here, we propose a method to learn soft tactile sensor membrane dynamics that accounts for sensor deformations caused by the physical interaction between the grasped object and environment. Our method combines the perceived 3D geometry of the membrane with proprioceptive reaction wrenches to predict future deformations conditioned on robot action. Grasped object poses are recovered from membrane geometry and reaction wrenches, decoupling interaction dynamics from the tactile observation model. We benchmark our approach on two real-world contact-rich tasks: drawing with a grasped marker and in-hand pivoting. Our results suggest that explicitly modeling membrane dynamics achieves better task performance and generalization to unseen objects than baselines.
APA
Oller, M., Lisbona, M.P.i., Berenson, D. & Fazeli, N.. (2023). Manipulation via Membranes: High-Resolution and Highly Deformable Tactile Sensing and Control. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:1850-1859 Available from https://proceedings.mlr.press/v205/oller23a.html.

Related Material