Visual Learning Towards Soft Robot Force Control using a 3D Metamaterial with Differential Stiffness

Fang Wan, Xiaobo Liu, Ning Guo, Xudong Han, Feng Tian, Chaoyang Song
Proceedings of the 5th Conference on Robot Learning, PMLR 164:1269-1278, 2022.

Abstract

This paper explores the feasibility of learning robot force control and interaction using soft metamaterial and machine vision. We start by investigating the differential stiffness of a hollow, cone-shaped, 3D metamaterial made from soft rubber, achieving a large stiffness ratio between the axial and radial directions that leads to an adaptive form response in omni-directions during physical interaction. Then, using image data collected from its internal deformation during various interactions, we explored two similar designs but different learning strategies to estimate force control and interactions on the end-effector of a UR10 e-series robot arm. One is to directly learn the force and torque response from raw images of the metamaterial’s internal deformation. The other is to indirectly estimate the 6D force and torque using a neural network by visually tracking the 6D pose of a marker fixed inside the 3D metamaterial. Finally, we integrated the two proposed systems and achieved similar force feedback and control interactions in simple tasks such as circle following and text writing. Our results show that the learning method holds the potential to support the concept of soft robot force control, providing an intuitive interface at a low cost for robotic systems, generating comparable and capable performances against classical force and torque sensors.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-wan22a, title = {Visual Learning Towards Soft Robot Force Control using a 3D Metamaterial with Differential Stiffness}, author = {Wan, Fang and Liu, Xiaobo and Guo, Ning and Han, Xudong and Tian, Feng and Song, Chaoyang}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {1269--1278}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/wan22a/wan22a.pdf}, url = {https://proceedings.mlr.press/v164/wan22a.html}, abstract = {This paper explores the feasibility of learning robot force control and interaction using soft metamaterial and machine vision. We start by investigating the differential stiffness of a hollow, cone-shaped, 3D metamaterial made from soft rubber, achieving a large stiffness ratio between the axial and radial directions that leads to an adaptive form response in omni-directions during physical interaction. Then, using image data collected from its internal deformation during various interactions, we explored two similar designs but different learning strategies to estimate force control and interactions on the end-effector of a UR10 e-series robot arm. One is to directly learn the force and torque response from raw images of the metamaterial’s internal deformation. The other is to indirectly estimate the 6D force and torque using a neural network by visually tracking the 6D pose of a marker fixed inside the 3D metamaterial. Finally, we integrated the two proposed systems and achieved similar force feedback and control interactions in simple tasks such as circle following and text writing. Our results show that the learning method holds the potential to support the concept of soft robot force control, providing an intuitive interface at a low cost for robotic systems, generating comparable and capable performances against classical force and torque sensors.} }
Endnote
%0 Conference Paper %T Visual Learning Towards Soft Robot Force Control using a 3D Metamaterial with Differential Stiffness %A Fang Wan %A Xiaobo Liu %A Ning Guo %A Xudong Han %A Feng Tian %A Chaoyang Song %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-wan22a %I PMLR %P 1269--1278 %U https://proceedings.mlr.press/v164/wan22a.html %V 164 %X This paper explores the feasibility of learning robot force control and interaction using soft metamaterial and machine vision. We start by investigating the differential stiffness of a hollow, cone-shaped, 3D metamaterial made from soft rubber, achieving a large stiffness ratio between the axial and radial directions that leads to an adaptive form response in omni-directions during physical interaction. Then, using image data collected from its internal deformation during various interactions, we explored two similar designs but different learning strategies to estimate force control and interactions on the end-effector of a UR10 e-series robot arm. One is to directly learn the force and torque response from raw images of the metamaterial’s internal deformation. The other is to indirectly estimate the 6D force and torque using a neural network by visually tracking the 6D pose of a marker fixed inside the 3D metamaterial. Finally, we integrated the two proposed systems and achieved similar force feedback and control interactions in simple tasks such as circle following and text writing. Our results show that the learning method holds the potential to support the concept of soft robot force control, providing an intuitive interface at a low cost for robotic systems, generating comparable and capable performances against classical force and torque sensors.
APA
Wan, F., Liu, X., Guo, N., Han, X., Tian, F. & Song, C.. (2022). Visual Learning Towards Soft Robot Force Control using a 3D Metamaterial with Differential Stiffness. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:1269-1278 Available from https://proceedings.mlr.press/v164/wan22a.html.

Related Material