Reinforcement Learning Enables Real-Time Planning and Control of Agile Maneuvers for Soft Robot Arms

Rianna Jitosho, Tyler Ga Wei Lum, Allison Okamura, Karen Liu
Proceedings of The 7th Conference on Robot Learning, PMLR 229:1131-1153, 2023.

Abstract

Control policies for soft robot arms typically assume quasi-static motion or require a hand-designed motion plan. To achieve real-time planning and control for tasks requiring highly dynamic maneuvers, we apply deep reinforcement learning to train a policy entirely in simulation, and we identify strategies and insights that bridge the gap between simulation and reality. In particular, we strengthen the policy’s tolerance for inaccuracies with domain randomization and implement crucial simulator modifications that improve actuation and sensor modeling, enabling zero-shot sim-to-real transfer without requiring high-fidelity soft robot dynamics. We demonstrate the effectiveness of this approach with experiments on physical hardware and show that our soft robot can reach target positions that require dynamic swinging motions. This is the first work to achieve such agile maneuvers on a physical soft robot, advancing the field of soft robot arm planning and control. Our code and videos are publicly available at https://sites.google.com/view/rl-soft-robot.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-jitosho23a, title = {Reinforcement Learning Enables Real-Time Planning and Control of Agile Maneuvers for Soft Robot Arms}, author = {Jitosho, Rianna and Lum, Tyler Ga Wei and Okamura, Allison and Liu, Karen}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {1131--1153}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/jitosho23a/jitosho23a.pdf}, url = {https://proceedings.mlr.press/v229/jitosho23a.html}, abstract = {Control policies for soft robot arms typically assume quasi-static motion or require a hand-designed motion plan. To achieve real-time planning and control for tasks requiring highly dynamic maneuvers, we apply deep reinforcement learning to train a policy entirely in simulation, and we identify strategies and insights that bridge the gap between simulation and reality. In particular, we strengthen the policy’s tolerance for inaccuracies with domain randomization and implement crucial simulator modifications that improve actuation and sensor modeling, enabling zero-shot sim-to-real transfer without requiring high-fidelity soft robot dynamics. We demonstrate the effectiveness of this approach with experiments on physical hardware and show that our soft robot can reach target positions that require dynamic swinging motions. This is the first work to achieve such agile maneuvers on a physical soft robot, advancing the field of soft robot arm planning and control. Our code and videos are publicly available at https://sites.google.com/view/rl-soft-robot.} }
Endnote
%0 Conference Paper %T Reinforcement Learning Enables Real-Time Planning and Control of Agile Maneuvers for Soft Robot Arms %A Rianna Jitosho %A Tyler Ga Wei Lum %A Allison Okamura %A Karen Liu %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-jitosho23a %I PMLR %P 1131--1153 %U https://proceedings.mlr.press/v229/jitosho23a.html %V 229 %X Control policies for soft robot arms typically assume quasi-static motion or require a hand-designed motion plan. To achieve real-time planning and control for tasks requiring highly dynamic maneuvers, we apply deep reinforcement learning to train a policy entirely in simulation, and we identify strategies and insights that bridge the gap between simulation and reality. In particular, we strengthen the policy’s tolerance for inaccuracies with domain randomization and implement crucial simulator modifications that improve actuation and sensor modeling, enabling zero-shot sim-to-real transfer without requiring high-fidelity soft robot dynamics. We demonstrate the effectiveness of this approach with experiments on physical hardware and show that our soft robot can reach target positions that require dynamic swinging motions. This is the first work to achieve such agile maneuvers on a physical soft robot, advancing the field of soft robot arm planning and control. Our code and videos are publicly available at https://sites.google.com/view/rl-soft-robot.
APA
Jitosho, R., Lum, T.G.W., Okamura, A. & Liu, K.. (2023). Reinforcement Learning Enables Real-Time Planning and Control of Agile Maneuvers for Soft Robot Arms. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:1131-1153 Available from https://proceedings.mlr.press/v229/jitosho23a.html.

Related Material