Learning the Dynamics of Compliant Tool-Environment Interaction for Visuo-Tactile Contact Servoing

Mark Van der Merwe, Dmitry Berenson, Nima Fazeli
Proceedings of The 6th Conference on Robot Learning, PMLR 205:2052-2061, 2023.

Abstract

Many manipulation tasks require the robot to control the contact between a grasped compliant tool and the environment, e.g. scraping a frying pan with a spatula. However, modeling tool-environment interaction is difficult, especially when the tool is compliant, and the robot cannot be expected to have the full geometry and physical properties (e.g., mass, stiffness, and friction) of all the tools it must use. We propose a framework that learns to predict the effects of a robot’s actions on the contact between the tool and the environment given visuo-tactile perception. Key to our framework is a novel contact feature representation that consists of a binary contact value, the line of contact, and an end-effector wrench. We propose a method to learn the dynamics of these contact features from real world data that does not require predicting the geometry of the compliant tool. We then propose a controller that uses this dynamics model for visuo-tactile contact servoing and show that it is effective at performing scraping tasks with a spatula, even in scenarios where precise contact needs to be made to avoid obstacles.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-merwe23a, title = {Learning the Dynamics of Compliant Tool-Environment Interaction for Visuo-Tactile Contact Servoing}, author = {Merwe, Mark Van der and Berenson, Dmitry and Fazeli, Nima}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {2052--2061}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/merwe23a/merwe23a.pdf}, url = {https://proceedings.mlr.press/v205/merwe23a.html}, abstract = {Many manipulation tasks require the robot to control the contact between a grasped compliant tool and the environment, e.g. scraping a frying pan with a spatula. However, modeling tool-environment interaction is difficult, especially when the tool is compliant, and the robot cannot be expected to have the full geometry and physical properties (e.g., mass, stiffness, and friction) of all the tools it must use. We propose a framework that learns to predict the effects of a robot’s actions on the contact between the tool and the environment given visuo-tactile perception. Key to our framework is a novel contact feature representation that consists of a binary contact value, the line of contact, and an end-effector wrench. We propose a method to learn the dynamics of these contact features from real world data that does not require predicting the geometry of the compliant tool. We then propose a controller that uses this dynamics model for visuo-tactile contact servoing and show that it is effective at performing scraping tasks with a spatula, even in scenarios where precise contact needs to be made to avoid obstacles.} }
Endnote
%0 Conference Paper %T Learning the Dynamics of Compliant Tool-Environment Interaction for Visuo-Tactile Contact Servoing %A Mark Van der Merwe %A Dmitry Berenson %A Nima Fazeli %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-merwe23a %I PMLR %P 2052--2061 %U https://proceedings.mlr.press/v205/merwe23a.html %V 205 %X Many manipulation tasks require the robot to control the contact between a grasped compliant tool and the environment, e.g. scraping a frying pan with a spatula. However, modeling tool-environment interaction is difficult, especially when the tool is compliant, and the robot cannot be expected to have the full geometry and physical properties (e.g., mass, stiffness, and friction) of all the tools it must use. We propose a framework that learns to predict the effects of a robot’s actions on the contact between the tool and the environment given visuo-tactile perception. Key to our framework is a novel contact feature representation that consists of a binary contact value, the line of contact, and an end-effector wrench. We propose a method to learn the dynamics of these contact features from real world data that does not require predicting the geometry of the compliant tool. We then propose a controller that uses this dynamics model for visuo-tactile contact servoing and show that it is effective at performing scraping tasks with a spatula, even in scenarios where precise contact needs to be made to avoid obstacles.
APA
Merwe, M.V.d., Berenson, D. & Fazeli, N.. (2023). Learning the Dynamics of Compliant Tool-Environment Interaction for Visuo-Tactile Contact Servoing. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:2052-2061 Available from https://proceedings.mlr.press/v205/merwe23a.html.

Related Material