Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real

Ofir Nachum, Michael Ahn, Hugo Ponte, Shixiang (Shane) Gu, Vikash Kumar
Proceedings of the Conference on Robot Learning, PMLR 100:110-121, 2020.

Abstract

Manipulation and locomotion are closely related problems that are often studied in isolation. In this work, we study the problem of coordinating multiple mobile agents to exhibit manipulation behaviors using a reinforcement learning (RL) approach. Our method hinges on the use of hierarchical sim2real – a simulated environment is used to learn low-level goal-reaching skills, which are then used as the action space for a high-level RL controller, also trained in simulation. The full hierarchical policy is then transferred to the real world in a zero-shot fashion. The application of domain randomization during training enables the learned behaviors to generalize to real-world settings, while the use of hierarchy provides a modular paradigm for learning and transferring increasingly complex behaviors. We evaluate our method on a number of real-world tasks, including coordinated object manipulation in a multi-agent setting.

Cite this Paper

BibTeX
@InProceedings{pmlr-v100-nachum20a, title = {Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real}, author = {Nachum, Ofir and Ahn, Michael and Ponte, Hugo and Gu, Shixiang (Shane) and Kumar, Vikash}, booktitle = {Proceedings of the Conference on Robot Learning}, pages = {110--121}, year = {2020}, editor = {Kaelbling, Leslie Pack and Kragic, Danica and Sugiura, Komei}, volume = {100}, series = {Proceedings of Machine Learning Research}, month = {30 Oct--01 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v100/nachum20a/nachum20a.pdf}, url = {https://proceedings.mlr.press/v100/nachum20a.html}, abstract = {Manipulation and locomotion are closely related problems that are often studied in isolation. In this work, we study the problem of coordinating multiple mobile agents to exhibit manipulation behaviors using a reinforcement learning (RL) approach. Our method hinges on the use of hierarchical sim2real – a simulated environment is used to learn low-level goal-reaching skills, which are then used as the action space for a high-level RL controller, also trained in simulation. The full hierarchical policy is then transferred to the real world in a zero-shot fashion. The application of domain randomization during training enables the learned behaviors to generalize to real-world settings, while the use of hierarchy provides a modular paradigm for learning and transferring increasingly complex behaviors. We evaluate our method on a number of real-world tasks, including coordinated object manipulation in a multi-agent setting.} }
Endnote
%0 Conference Paper %T Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real %A Ofir Nachum %A Michael Ahn %A Hugo Ponte %A Shixiang (Shane) Gu %A Vikash Kumar %B Proceedings of the Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2020 %E Leslie Pack Kaelbling %E Danica Kragic %E Komei Sugiura %F pmlr-v100-nachum20a %I PMLR %P 110--121 %U https://proceedings.mlr.press/v100/nachum20a.html %V 100 %X Manipulation and locomotion are closely related problems that are often studied in isolation. In this work, we study the problem of coordinating multiple mobile agents to exhibit manipulation behaviors using a reinforcement learning (RL) approach. Our method hinges on the use of hierarchical sim2real – a simulated environment is used to learn low-level goal-reaching skills, which are then used as the action space for a high-level RL controller, also trained in simulation. The full hierarchical policy is then transferred to the real world in a zero-shot fashion. The application of domain randomization during training enables the learned behaviors to generalize to real-world settings, while the use of hierarchy provides a modular paradigm for learning and transferring increasingly complex behaviors. We evaluate our method on a number of real-world tasks, including coordinated object manipulation in a multi-agent setting.
APA
Nachum, O., Ahn, M., Ponte, H., Gu, S.(. & Kumar, V.. (2020). Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real. Proceedings of the Conference on Robot Learning, in Proceedings of Machine Learning Research 100:110-121 Available from https://proceedings.mlr.press/v100/nachum20a.html.

Related Material