Twisting Lids Off with Two Hands

Toru Lin, Zhao-Heng Yin, Haozhi Qi, Pieter Abbeel, Jitendra Malik
Proceedings of The 8th Conference on Robot Learning, PMLR 270:5220-5235, 2025.

Abstract

Manipulating objects with two multi-fingered hands has been a long-standing challenge in robotics, due to the contact-rich nature of many manipulation tasks and the complexity inherent in coordinating a high-dimensional bimanual system. In this work, we share novel insights into physical modeling, real-time perception, and reward design that enable policies trained in simulation using deep reinforcement learning (RL) to be effectively and efficiently transferred to the real world. Specifically, we consider the problem of twisting lids of various bottle-like objects with two hands, demonstrating policies with generalization capabilities across a diverse set of unseen objects as well as dynamic and dexterous behaviors. To the best of our knowledge, this is the first sim-to-real RL system that enables such capabilities on bimanual multi-fingered hands.

Cite this Paper

BibTeX
@InProceedings{pmlr-v270-lin25c, title = {Twisting Lids Off with Two Hands}, author = {Lin, Toru and Yin, Zhao-Heng and Qi, Haozhi and Abbeel, Pieter and Malik, Jitendra}, booktitle = {Proceedings of The 8th Conference on Robot Learning}, pages = {5220--5235}, year = {2025}, editor = {Agrawal, Pulkit and Kroemer, Oliver and Burgard, Wolfram}, volume = {270}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v270/main/assets/lin25c/lin25c.pdf}, url = {https://proceedings.mlr.press/v270/lin25c.html}, abstract = {Manipulating objects with two multi-fingered hands has been a long-standing challenge in robotics, due to the contact-rich nature of many manipulation tasks and the complexity inherent in coordinating a high-dimensional bimanual system. In this work, we share novel insights into physical modeling, real-time perception, and reward design that enable policies trained in simulation using deep reinforcement learning (RL) to be effectively and efficiently transferred to the real world. Specifically, we consider the problem of twisting lids of various bottle-like objects with two hands, demonstrating policies with generalization capabilities across a diverse set of unseen objects as well as dynamic and dexterous behaviors. To the best of our knowledge, this is the first sim-to-real RL system that enables such capabilities on bimanual multi-fingered hands.} }
Endnote
%0 Conference Paper %T Twisting Lids Off with Two Hands %A Toru Lin %A Zhao-Heng Yin %A Haozhi Qi %A Pieter Abbeel %A Jitendra Malik %B Proceedings of The 8th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Pulkit Agrawal %E Oliver Kroemer %E Wolfram Burgard %F pmlr-v270-lin25c %I PMLR %P 5220--5235 %U https://proceedings.mlr.press/v270/lin25c.html %V 270 %X Manipulating objects with two multi-fingered hands has been a long-standing challenge in robotics, due to the contact-rich nature of many manipulation tasks and the complexity inherent in coordinating a high-dimensional bimanual system. In this work, we share novel insights into physical modeling, real-time perception, and reward design that enable policies trained in simulation using deep reinforcement learning (RL) to be effectively and efficiently transferred to the real world. Specifically, we consider the problem of twisting lids of various bottle-like objects with two hands, demonstrating policies with generalization capabilities across a diverse set of unseen objects as well as dynamic and dexterous behaviors. To the best of our knowledge, this is the first sim-to-real RL system that enables such capabilities on bimanual multi-fingered hands.
APA
Lin, T., Yin, Z., Qi, H., Abbeel, P. & Malik, J.. (2025). Twisting Lids Off with Two Hands. Proceedings of The 8th Conference on Robot Learning, in Proceedings of Machine Learning Research 270:5220-5235 Available from https://proceedings.mlr.press/v270/lin25c.html.

Related Material