Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real World

Nico Gürtler, Felix Widmaier, Cansu Sancaktar, Sebastian Blaes, Pavel Kolev, Stefan Bauer, Manuel Wüthrich, Markus Wulfmeier, Martin Riedmiller, Arthur Allshire, Qiang Wang, Robert McCarthy, Hangyeol Kim, Jongchan Baek, Wookyong Kwon, Shanliang Qian, Yasunori Toshimitsu, Mike Yan Michelis, Amirhossein Kazemipour, Arman Raayatsanati, Hehui Zheng, Barnabas Gavin Cangan, Bernhard Schölkopf, Georg Martius
Proceedings of the NeurIPS 2022 Competitions Track, PMLR 220:133-150, 2022.

Abstract

Experimentation on real robots is demanding in terms of time and costs. For this reason, a large part of the reinforcement learning (RL) community uses simulators to develop and benchmark algorithms. However, insights gained in simulation do not necessarily translate to real robots, in particular for tasks involving complex interactions with the environment. The Real Robot Challenge 2022 therefore served as a bridge between the RL and robotics communities by allowing participants to experiment remotely with a real robot - as easily as in simulation. In the last years, offline reinforcement learning has matured into a promising paradigm for learning from pre-collected datasets, alleviating the reliance on expensive online interactions. We therefore asked the participants to learn two dexterous manipulation tasks involving pushing, grasping, and in-hand orientation from provided real-robot datasets. An extensive software documentation and an initial stage based on a simulation of the real set-up made the competition particularly accessible. By giving each team plenty of access budget to evaluate their offline-learned policies on a cluster of seven identical real TriFinger platforms, we organized an exciting competition for machine learners and roboticists alike. In this work we state the rules of the competition, present the methods used by the winning teams and compare their results with a benchmark of state-of-the-art offline RL algorithms on the challenge datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v220-gurtler23a, title = {Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real World}, author = {G\"urtler, Nico and Widmaier, Felix and Sancaktar, Cansu and Blaes, Sebastian and Kolev, Pavel and Bauer, Stefan and W\"uthrich, Manuel and Wulfmeier, Markus and Riedmiller, Martin and Allshire, Arthur and Wang, Qiang and McCarthy, Robert and Kim, Hangyeol and Baek, Jongchan and Kwon, Wookyong and Qian, Shanliang and Toshimitsu, Yasunori and Michelis, Mike Yan and Kazemipour, Amirhossein and Raayatsanati, Arman and Zheng, Hehui and Cangan, Barnabas Gavin and Sch\"olkopf, Bernhard and Martius, Georg}, booktitle = {Proceedings of the NeurIPS 2022 Competitions Track}, pages = {133--150}, year = {2022}, editor = {Ciccone, Marco and Stolovitzky, Gustavo and Albrecht, Jacob}, volume = {220}, series = {Proceedings of Machine Learning Research}, month = {28 Nov--09 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v220/gurtler23a/gurtler23a.pdf}, url = {https://proceedings.mlr.press/v220/gurtler23a.html}, abstract = {Experimentation on real robots is demanding in terms of time and costs. For this reason, a large part of the reinforcement learning (RL) community uses simulators to develop and benchmark algorithms. However, insights gained in simulation do not necessarily translate to real robots, in particular for tasks involving complex interactions with the environment. The Real Robot Challenge 2022 therefore served as a bridge between the RL and robotics communities by allowing participants to experiment remotely with a real robot - as easily as in simulation. In the last years, offline reinforcement learning has matured into a promising paradigm for learning from pre-collected datasets, alleviating the reliance on expensive online interactions. We therefore asked the participants to learn two dexterous manipulation tasks involving pushing, grasping, and in-hand orientation from provided real-robot datasets. An extensive software documentation and an initial stage based on a simulation of the real set-up made the competition particularly accessible. By giving each team plenty of access budget to evaluate their offline-learned policies on a cluster of seven identical real TriFinger platforms, we organized an exciting competition for machine learners and roboticists alike. In this work we state the rules of the competition, present the methods used by the winning teams and compare their results with a benchmark of state-of-the-art offline RL algorithms on the challenge datasets.} }
Endnote
%0 Conference Paper %T Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real World %A Nico Gürtler %A Felix Widmaier %A Cansu Sancaktar %A Sebastian Blaes %A Pavel Kolev %A Stefan Bauer %A Manuel Wüthrich %A Markus Wulfmeier %A Martin Riedmiller %A Arthur Allshire %A Qiang Wang %A Robert McCarthy %A Hangyeol Kim %A Jongchan Baek %A Wookyong Kwon %A Shanliang Qian %A Yasunori Toshimitsu %A Mike Yan Michelis %A Amirhossein Kazemipour %A Arman Raayatsanati %A Hehui Zheng %A Barnabas Gavin Cangan %A Bernhard Schölkopf %A Georg Martius %B Proceedings of the NeurIPS 2022 Competitions Track %C Proceedings of Machine Learning Research %D 2022 %E Marco Ciccone %E Gustavo Stolovitzky %E Jacob Albrecht %F pmlr-v220-gurtler23a %I PMLR %P 133--150 %U https://proceedings.mlr.press/v220/gurtler23a.html %V 220 %X Experimentation on real robots is demanding in terms of time and costs. For this reason, a large part of the reinforcement learning (RL) community uses simulators to develop and benchmark algorithms. However, insights gained in simulation do not necessarily translate to real robots, in particular for tasks involving complex interactions with the environment. The Real Robot Challenge 2022 therefore served as a bridge between the RL and robotics communities by allowing participants to experiment remotely with a real robot - as easily as in simulation. In the last years, offline reinforcement learning has matured into a promising paradigm for learning from pre-collected datasets, alleviating the reliance on expensive online interactions. We therefore asked the participants to learn two dexterous manipulation tasks involving pushing, grasping, and in-hand orientation from provided real-robot datasets. An extensive software documentation and an initial stage based on a simulation of the real set-up made the competition particularly accessible. By giving each team plenty of access budget to evaluate their offline-learned policies on a cluster of seven identical real TriFinger platforms, we organized an exciting competition for machine learners and roboticists alike. In this work we state the rules of the competition, present the methods used by the winning teams and compare their results with a benchmark of state-of-the-art offline RL algorithms on the challenge datasets.
APA
Gürtler, N., Widmaier, F., Sancaktar, C., Blaes, S., Kolev, P., Bauer, S., Wüthrich, M., Wulfmeier, M., Riedmiller, M., Allshire, A., Wang, Q., McCarthy, R., Kim, H., Baek, J., Kwon, W., Qian, S., Toshimitsu, Y., Michelis, M.Y., Kazemipour, A., Raayatsanati, A., Zheng, H., Cangan, B.G., Schölkopf, B. & Martius, G.. (2022). Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real World. Proceedings of the NeurIPS 2022 Competitions Track, in Proceedings of Machine Learning Research 220:133-150 Available from https://proceedings.mlr.press/v220/gurtler23a.html.

Related Material