Efficient Sim-to-real Transfer of Contact-Rich Manipulation Skills with Online Admittance Residual Learning

Xiang Zhang, Changhao Wang, Lingfeng Sun, Zheng Wu, Xinghao Zhu, Masayoshi Tomizuka
Proceedings of The 7th Conference on Robot Learning, PMLR 229:1621-1639, 2023.

Abstract

Learning contact-rich manipulation skills is essential. Such skills require the robots to interact with the environment with feasible manipulation trajectories and suitable compliance control parameters to enable safe and stable contact. However, learning these skills is challenging due to data inefficiency in the real world and the sim-to-real gap in simulation. In this paper, we introduce a hybrid offline-online framework to learn robust manipulation skills. We employ model-free reinforcement learning for the offline phase to obtain the robot motion and compliance control parameters in simulation \RV{with domain randomization}. Subsequently, in the online phase, we learn the residual of the compliance control parameters to maximize robot performance-related criteria with force sensor measurements in real-time. To demonstrate the effectiveness and robustness of our approach, we provide comparative results against existing methods for assembly, pivoting, and screwing tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-zhang23e, title = {Efficient Sim-to-real Transfer of Contact-Rich Manipulation Skills with Online Admittance Residual Learning}, author = {Zhang, Xiang and Wang, Changhao and Sun, Lingfeng and Wu, Zheng and Zhu, Xinghao and Tomizuka, Masayoshi}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {1621--1639}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/zhang23e/zhang23e.pdf}, url = {https://proceedings.mlr.press/v229/zhang23e.html}, abstract = {Learning contact-rich manipulation skills is essential. Such skills require the robots to interact with the environment with feasible manipulation trajectories and suitable compliance control parameters to enable safe and stable contact. However, learning these skills is challenging due to data inefficiency in the real world and the sim-to-real gap in simulation. In this paper, we introduce a hybrid offline-online framework to learn robust manipulation skills. We employ model-free reinforcement learning for the offline phase to obtain the robot motion and compliance control parameters in simulation \RV{with domain randomization}. Subsequently, in the online phase, we learn the residual of the compliance control parameters to maximize robot performance-related criteria with force sensor measurements in real-time. To demonstrate the effectiveness and robustness of our approach, we provide comparative results against existing methods for assembly, pivoting, and screwing tasks.} }
Endnote
%0 Conference Paper %T Efficient Sim-to-real Transfer of Contact-Rich Manipulation Skills with Online Admittance Residual Learning %A Xiang Zhang %A Changhao Wang %A Lingfeng Sun %A Zheng Wu %A Xinghao Zhu %A Masayoshi Tomizuka %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-zhang23e %I PMLR %P 1621--1639 %U https://proceedings.mlr.press/v229/zhang23e.html %V 229 %X Learning contact-rich manipulation skills is essential. Such skills require the robots to interact with the environment with feasible manipulation trajectories and suitable compliance control parameters to enable safe and stable contact. However, learning these skills is challenging due to data inefficiency in the real world and the sim-to-real gap in simulation. In this paper, we introduce a hybrid offline-online framework to learn robust manipulation skills. We employ model-free reinforcement learning for the offline phase to obtain the robot motion and compliance control parameters in simulation \RV{with domain randomization}. Subsequently, in the online phase, we learn the residual of the compliance control parameters to maximize robot performance-related criteria with force sensor measurements in real-time. To demonstrate the effectiveness and robustness of our approach, we provide comparative results against existing methods for assembly, pivoting, and screwing tasks.
APA
Zhang, X., Wang, C., Sun, L., Wu, Z., Zhu, X. & Tomizuka, M.. (2023). Efficient Sim-to-real Transfer of Contact-Rich Manipulation Skills with Online Admittance Residual Learning. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:1621-1639 Available from https://proceedings.mlr.press/v229/zhang23e.html.

Related Material