Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion

Zipeng Fu, Xuxin Cheng, Deepak Pathak
Proceedings of The 6th Conference on Robot Learning, PMLR 205:138-149, 2023.

Abstract

An attached arm can significantly increase the applicability of legged robots to several mobile manipulation tasks that are not possible for the wheeled or tracked counterparts. The standard modular control pipeline for such legged manipulators is to decouple the controller into that of manipulation and locomotion. However, this is ineffective. It requires immense engineering to support coordination between the arm and legs, and error can propagate across modules causing non-smooth unnatural motions. It is also biological implausible given evidence for strong motor synergies across limbs. In this work, we propose to learn a unified policy for whole-body control of a legged manipulator using reinforcement learning. We propose Regularized Online Adaptation to bridge the Sim2Real gap for high-DoF control, and Advantage Mixing exploiting the causal dependency in the action space to overcome local minima during training the whole-body system. We also present a simple design for a low-cost legged manipulator, and find that our unified policy can demonstrate dynamic and agile behaviors across several task setups. Videos are at https://maniploco.github.io

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-fu23a, title = {Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion}, author = {Fu, Zipeng and Cheng, Xuxin and Pathak, Deepak}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {138--149}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/fu23a/fu23a.pdf}, url = {https://proceedings.mlr.press/v205/fu23a.html}, abstract = {An attached arm can significantly increase the applicability of legged robots to several mobile manipulation tasks that are not possible for the wheeled or tracked counterparts. The standard modular control pipeline for such legged manipulators is to decouple the controller into that of manipulation and locomotion. However, this is ineffective. It requires immense engineering to support coordination between the arm and legs, and error can propagate across modules causing non-smooth unnatural motions. It is also biological implausible given evidence for strong motor synergies across limbs. In this work, we propose to learn a unified policy for whole-body control of a legged manipulator using reinforcement learning. We propose Regularized Online Adaptation to bridge the Sim2Real gap for high-DoF control, and Advantage Mixing exploiting the causal dependency in the action space to overcome local minima during training the whole-body system. We also present a simple design for a low-cost legged manipulator, and find that our unified policy can demonstrate dynamic and agile behaviors across several task setups. Videos are at https://maniploco.github.io} }
Endnote
%0 Conference Paper %T Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion %A Zipeng Fu %A Xuxin Cheng %A Deepak Pathak %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-fu23a %I PMLR %P 138--149 %U https://proceedings.mlr.press/v205/fu23a.html %V 205 %X An attached arm can significantly increase the applicability of legged robots to several mobile manipulation tasks that are not possible for the wheeled or tracked counterparts. The standard modular control pipeline for such legged manipulators is to decouple the controller into that of manipulation and locomotion. However, this is ineffective. It requires immense engineering to support coordination between the arm and legs, and error can propagate across modules causing non-smooth unnatural motions. It is also biological implausible given evidence for strong motor synergies across limbs. In this work, we propose to learn a unified policy for whole-body control of a legged manipulator using reinforcement learning. We propose Regularized Online Adaptation to bridge the Sim2Real gap for high-DoF control, and Advantage Mixing exploiting the causal dependency in the action space to overcome local minima during training the whole-body system. We also present a simple design for a low-cost legged manipulator, and find that our unified policy can demonstrate dynamic and agile behaviors across several task setups. Videos are at https://maniploco.github.io
APA
Fu, Z., Cheng, X. & Pathak, D.. (2023). Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:138-149 Available from https://proceedings.mlr.press/v205/fu23a.html.

Related Material