Continuous Versatile Jumping Using Learned Action Residuals

Yuxiang Yang, Xiangyun Meng, Wenhao Yu, Tingnan Zhang, Jie Tan, Byron Boots
Proceedings of The 5th Annual Learning for Dynamics and Control Conference, PMLR 211:770-782, 2023.

Abstract

Jumping is essential for legged robots to traverse through difficult terrains. In this work, we propose a hierarchical framework that combines optimal control and reinforcement learning to learn continuous jumping motions for quadrupedal robots. The core of our framework is the high-level stance controller, which combines a manually designed acceleration controller with a learned residual policy. As the acceleration controller warm starts policy for efficient and smooth training, the trained policy improves the overall jumping stability beyond the controller’s limitations. In addition, a low-level whole-body controller converts the body pose command from the stance controller to motor actions. After training in simulation, our framework can be deployed directly to the real robot, and perform versatile, continuous jumping motions, including omni-directional jumps at up to 50cm high, 60cm forward, and jump-turning at up to 90 degrees. Please visit our website for more results: https://sites.google.com/view/learning-to-jump.

Cite this Paper

BibTeX
@InProceedings{pmlr-v211-yang23b, title = {Continuous Versatile Jumping Using Learned Action Residuals}, author = {Yang, Yuxiang and Meng, Xiangyun and Yu, Wenhao and Zhang, Tingnan and Tan, Jie and Boots, Byron}, booktitle = {Proceedings of The 5th Annual Learning for Dynamics and Control Conference}, pages = {770--782}, year = {2023}, editor = {Matni, Nikolai and Morari, Manfred and Pappas, George J.}, volume = {211}, series = {Proceedings of Machine Learning Research}, month = {15--16 Jun}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v211/yang23b/yang23b.pdf}, url = {https://proceedings.mlr.press/v211/yang23b.html}, abstract = {Jumping is essential for legged robots to traverse through difficult terrains. In this work, we propose a hierarchical framework that combines optimal control and reinforcement learning to learn continuous jumping motions for quadrupedal robots. The core of our framework is the high-level stance controller, which combines a manually designed acceleration controller with a learned residual policy. As the acceleration controller warm starts policy for efficient and smooth training, the trained policy improves the overall jumping stability beyond the controller’s limitations. In addition, a low-level whole-body controller converts the body pose command from the stance controller to motor actions. After training in simulation, our framework can be deployed directly to the real robot, and perform versatile, continuous jumping motions, including omni-directional jumps at up to 50cm high, 60cm forward, and jump-turning at up to 90 degrees. Please visit our website for more results: https://sites.google.com/view/learning-to-jump.} }
Endnote
%0 Conference Paper %T Continuous Versatile Jumping Using Learned Action Residuals %A Yuxiang Yang %A Xiangyun Meng %A Wenhao Yu %A Tingnan Zhang %A Jie Tan %A Byron Boots %B Proceedings of The 5th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2023 %E Nikolai Matni %E Manfred Morari %E George J. Pappas %F pmlr-v211-yang23b %I PMLR %P 770--782 %U https://proceedings.mlr.press/v211/yang23b.html %V 211 %X Jumping is essential for legged robots to traverse through difficult terrains. In this work, we propose a hierarchical framework that combines optimal control and reinforcement learning to learn continuous jumping motions for quadrupedal robots. The core of our framework is the high-level stance controller, which combines a manually designed acceleration controller with a learned residual policy. As the acceleration controller warm starts policy for efficient and smooth training, the trained policy improves the overall jumping stability beyond the controller’s limitations. In addition, a low-level whole-body controller converts the body pose command from the stance controller to motor actions. After training in simulation, our framework can be deployed directly to the real robot, and perform versatile, continuous jumping motions, including omni-directional jumps at up to 50cm high, 60cm forward, and jump-turning at up to 90 degrees. Please visit our website for more results: https://sites.google.com/view/learning-to-jump.
APA
Yang, Y., Meng, X., Yu, W., Zhang, T., Tan, J. & Boots, B.. (2023). Continuous Versatile Jumping Using Learned Action Residuals. Proceedings of The 5th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 211:770-782 Available from https://proceedings.mlr.press/v211/yang23b.html.

Related Material