MyoChallenge 2022: Learning contact-rich manipulation using a musculoskeletal hand

Vittorio Caggiano, Guillaume Durandau, Huwawei Wang, Alberto Chiappa, Alexander Mathis, Pablo Tano, Nisheet Patel, Alexandre Pouget, Pierre Schumacher, Georg Martius, Daniel Haeufle, Yiran Geng, Boshi An, Yifan Zhong, Jiaming Ji, Yuanpei Chen, Hao Dong, Yaodong Yang, Rahul Siripurapu, Luis Eduardo Ferro Diez, Michael Kopp, Vihang Patil, Sepp Hochreiter, Yuval Tassa, Josh Merel, Randy Schultheis, Seungmoon Song, Massimo Sartori, Vikash Kumar
Proceedings of the NeurIPS 2022 Competitions Track, PMLR 220:233-250, 2022.

Abstract

Manual dexterity has been considered one of the critical components for human evolution. The ability to perform movements as simple as holding and rotating an object in the hand without dropping it needs the coordination of more than 35 muscles which act synergistically or antagonistically on multiple joints. This complexity in control is markedly different from typical pre-specified movements or torque based controls used in robotics. In the MyoChallenge at the NeurIPS 2022 competition track, we challenged the community to develop controllers for a realistic hand to solve a series of dexterous manipulation tasks. The MyoSuite framework was used to train and test controllers on realistic, contact rich and computation efficient virtual neuromusculoskeletal model of the hand and wrist. Two tasks were proposed: a die re-orientation and a boading ball (rotation of two spheres respect to each other) tasks. More than 40 teams participated to the challenge and submitted more than 340 solutions. The challenge was split in two phases. In the first phase, where a limited set of objectives and randomization were proposed, teams managed to achieve high performance, in particular in the boading-ball task. In the second phase as the focus shifted towards generalization of task solutions to extensive variations of object and task properties, teams saw significant performance drop. This shows that there is still a large gap in developing agents capable of generalizable skilled manipulation. In future challenges, we will continue pursuing the generalizability both in skills and agility of the tasks exploring additional realistic neuromusculoskeletal models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v220-caggiano23a, title = {MyoChallenge 2022: Learning contact-rich manipulation using a musculoskeletal hand}, author = {Caggiano, Vittorio and Durandau, Guillaume and Wang, Huwawei and Chiappa, Alberto and Mathis, Alexander and Tano, Pablo and Patel, Nisheet and Pouget, Alexandre and Schumacher, Pierre and Martius, Georg and Haeufle, Daniel and Geng, Yiran and An, Boshi and Zhong, Yifan and Ji, Jiaming and Chen, Yuanpei and Dong, Hao and Yang, Yaodong and Siripurapu, Rahul and Ferro Diez, Luis Eduardo and Kopp, Michael and Patil, Vihang and Hochreiter, Sepp and Tassa, Yuval and Merel, Josh and Schultheis, Randy and Song, Seungmoon and Sartori, Massimo and Kumar, Vikash}, booktitle = {Proceedings of the NeurIPS 2022 Competitions Track}, pages = {233--250}, year = {2022}, editor = {Ciccone, Marco and Stolovitzky, Gustavo and Albrecht, Jacob}, volume = {220}, series = {Proceedings of Machine Learning Research}, month = {28 Nov--09 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v220/caggiano23a/caggiano23a.pdf}, url = {https://proceedings.mlr.press/v220/caggiano23a.html}, abstract = {Manual dexterity has been considered one of the critical components for human evolution. The ability to perform movements as simple as holding and rotating an object in the hand without dropping it needs the coordination of more than 35 muscles which act synergistically or antagonistically on multiple joints. This complexity in control is markedly different from typical pre-specified movements or torque based controls used in robotics. In the MyoChallenge at the NeurIPS 2022 competition track, we challenged the community to develop controllers for a realistic hand to solve a series of dexterous manipulation tasks. The MyoSuite framework was used to train and test controllers on realistic, contact rich and computation efficient virtual neuromusculoskeletal model of the hand and wrist. Two tasks were proposed: a die re-orientation and a boading ball (rotation of two spheres respect to each other) tasks. More than 40 teams participated to the challenge and submitted more than 340 solutions. The challenge was split in two phases. In the first phase, where a limited set of objectives and randomization were proposed, teams managed to achieve high performance, in particular in the boading-ball task. In the second phase as the focus shifted towards generalization of task solutions to extensive variations of object and task properties, teams saw significant performance drop. This shows that there is still a large gap in developing agents capable of generalizable skilled manipulation. In future challenges, we will continue pursuing the generalizability both in skills and agility of the tasks exploring additional realistic neuromusculoskeletal models.} }
Endnote
%0 Conference Paper %T MyoChallenge 2022: Learning contact-rich manipulation using a musculoskeletal hand %A Vittorio Caggiano %A Guillaume Durandau %A Huwawei Wang %A Alberto Chiappa %A Alexander Mathis %A Pablo Tano %A Nisheet Patel %A Alexandre Pouget %A Pierre Schumacher %A Georg Martius %A Daniel Haeufle %A Yiran Geng %A Boshi An %A Yifan Zhong %A Jiaming Ji %A Yuanpei Chen %A Hao Dong %A Yaodong Yang %A Rahul Siripurapu %A Luis Eduardo Ferro Diez %A Michael Kopp %A Vihang Patil %A Sepp Hochreiter %A Yuval Tassa %A Josh Merel %A Randy Schultheis %A Seungmoon Song %A Massimo Sartori %A Vikash Kumar %B Proceedings of the NeurIPS 2022 Competitions Track %C Proceedings of Machine Learning Research %D 2022 %E Marco Ciccone %E Gustavo Stolovitzky %E Jacob Albrecht %F pmlr-v220-caggiano23a %I PMLR %P 233--250 %U https://proceedings.mlr.press/v220/caggiano23a.html %V 220 %X Manual dexterity has been considered one of the critical components for human evolution. The ability to perform movements as simple as holding and rotating an object in the hand without dropping it needs the coordination of more than 35 muscles which act synergistically or antagonistically on multiple joints. This complexity in control is markedly different from typical pre-specified movements or torque based controls used in robotics. In the MyoChallenge at the NeurIPS 2022 competition track, we challenged the community to develop controllers for a realistic hand to solve a series of dexterous manipulation tasks. The MyoSuite framework was used to train and test controllers on realistic, contact rich and computation efficient virtual neuromusculoskeletal model of the hand and wrist. Two tasks were proposed: a die re-orientation and a boading ball (rotation of two spheres respect to each other) tasks. More than 40 teams participated to the challenge and submitted more than 340 solutions. The challenge was split in two phases. In the first phase, where a limited set of objectives and randomization were proposed, teams managed to achieve high performance, in particular in the boading-ball task. In the second phase as the focus shifted towards generalization of task solutions to extensive variations of object and task properties, teams saw significant performance drop. This shows that there is still a large gap in developing agents capable of generalizable skilled manipulation. In future challenges, we will continue pursuing the generalizability both in skills and agility of the tasks exploring additional realistic neuromusculoskeletal models.
APA
Caggiano, V., Durandau, G., Wang, H., Chiappa, A., Mathis, A., Tano, P., Patel, N., Pouget, A., Schumacher, P., Martius, G., Haeufle, D., Geng, Y., An, B., Zhong, Y., Ji, J., Chen, Y., Dong, H., Yang, Y., Siripurapu, R., Ferro Diez, L.E., Kopp, M., Patil, V., Hochreiter, S., Tassa, Y., Merel, J., Schultheis, R., Song, S., Sartori, M. & Kumar, V.. (2022). MyoChallenge 2022: Learning contact-rich manipulation using a musculoskeletal hand. Proceedings of the NeurIPS 2022 Competitions Track, in Proceedings of Machine Learning Research 220:233-250 Available from https://proceedings.mlr.press/v220/caggiano23a.html.

Related Material