Reinforcement learning-driven parametric curve fitting for snake robot gait design

Jack Naish, Jacob Rodriguez, Jenny Zhang, Bryson Jones, Guglielmo Daddi, Andrew Orekhov, Rob Royce, Michael Paton, Howie Choset, Masahiro Ono, Rohan Thakker
Proceedings of the 6th Annual Learning for Dynamics & Control Conference, PMLR 242:1715-1727, 2024.

Abstract

Snake-inspired robots demonstrate exceptional versatility through challenging terrains such as sand, rubble, and ice. However, their high-dimensional continuous action spaces make analytical gait design challenging. Early works by Hirose (1994) showed that gait parameterization over low-dimensional spatially and temporally varying sine waves can serve as basis functions for the shape-space or central pattern generators (CPGs). Recent approaches to designing CPGs have combined annealed chain-fitting, which solves for joint angles that fit a snake robot to a desired backbone curve, and keyframe extraction, which then fits analytic shape functions to the resulting optimized joint angles. However, the non-convex optimization associated with these methods is fraught with local optima exacerbated by constraints such as actuator limits. Reinforcement Learning has emerged as a promising alternative for searching over such spaces. However, end-to-end RL approaches trained purely in simulation are vulnerable to reality distribution shifts, lack safety guarantees, and don’t yield an intuitive representation of the learned gait. We propose a method that translates a gait found via policy search into a parametric representation of its component sinusoidal equations thus leveraging the strengths of both learning-based and classical approaches. Simulation and hardware experiments show that the proposed pipeline can generate parametric gaits where classical curve fitting-based approaches fail.

Cite this Paper

BibTeX
@InProceedings{pmlr-v242-naish24a, title = {Reinforcement learning-driven parametric curve fitting for snake robot gait design}, author = {Naish, Jack and Rodriguez, Jacob and Zhang, Jenny and Jones, Bryson and Daddi, Guglielmo and Orekhov, Andrew and Royce, Rob and Paton, Michael and Choset, Howie and Ono, Masahiro and Thakker, Rohan}, booktitle = {Proceedings of the 6th Annual Learning for Dynamics & Control Conference}, pages = {1715--1727}, year = {2024}, editor = {Abate, Alessandro and Cannon, Mark and Margellos, Kostas and Papachristodoulou, Antonis}, volume = {242}, series = {Proceedings of Machine Learning Research}, month = {15--17 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v242/naish24a/naish24a.pdf}, url = {https://proceedings.mlr.press/v242/naish24a.html}, abstract = {Snake-inspired robots demonstrate exceptional versatility through challenging terrains such as sand, rubble, and ice. However, their high-dimensional continuous action spaces make analytical gait design challenging. Early works by Hirose (1994) showed that gait parameterization over low-dimensional spatially and temporally varying sine waves can serve as basis functions for the shape-space or central pattern generators (CPGs). Recent approaches to designing CPGs have combined annealed chain-fitting, which solves for joint angles that fit a snake robot to a desired backbone curve, and keyframe extraction, which then fits analytic shape functions to the resulting optimized joint angles. However, the non-convex optimization associated with these methods is fraught with local optima exacerbated by constraints such as actuator limits. Reinforcement Learning has emerged as a promising alternative for searching over such spaces. However, end-to-end RL approaches trained purely in simulation are vulnerable to reality distribution shifts, lack safety guarantees, and don’t yield an intuitive representation of the learned gait. We propose a method that translates a gait found via policy search into a parametric representation of its component sinusoidal equations thus leveraging the strengths of both learning-based and classical approaches. Simulation and hardware experiments show that the proposed pipeline can generate parametric gaits where classical curve fitting-based approaches fail.} }
Endnote
%0 Conference Paper %T Reinforcement learning-driven parametric curve fitting for snake robot gait design %A Jack Naish %A Jacob Rodriguez %A Jenny Zhang %A Bryson Jones %A Guglielmo Daddi %A Andrew Orekhov %A Rob Royce %A Michael Paton %A Howie Choset %A Masahiro Ono %A Rohan Thakker %B Proceedings of the 6th Annual Learning for Dynamics & Control Conference %C Proceedings of Machine Learning Research %D 2024 %E Alessandro Abate %E Mark Cannon %E Kostas Margellos %E Antonis Papachristodoulou %F pmlr-v242-naish24a %I PMLR %P 1715--1727 %U https://proceedings.mlr.press/v242/naish24a.html %V 242 %X Snake-inspired robots demonstrate exceptional versatility through challenging terrains such as sand, rubble, and ice. However, their high-dimensional continuous action spaces make analytical gait design challenging. Early works by Hirose (1994) showed that gait parameterization over low-dimensional spatially and temporally varying sine waves can serve as basis functions for the shape-space or central pattern generators (CPGs). Recent approaches to designing CPGs have combined annealed chain-fitting, which solves for joint angles that fit a snake robot to a desired backbone curve, and keyframe extraction, which then fits analytic shape functions to the resulting optimized joint angles. However, the non-convex optimization associated with these methods is fraught with local optima exacerbated by constraints such as actuator limits. Reinforcement Learning has emerged as a promising alternative for searching over such spaces. However, end-to-end RL approaches trained purely in simulation are vulnerable to reality distribution shifts, lack safety guarantees, and don’t yield an intuitive representation of the learned gait. We propose a method that translates a gait found via policy search into a parametric representation of its component sinusoidal equations thus leveraging the strengths of both learning-based and classical approaches. Simulation and hardware experiments show that the proposed pipeline can generate parametric gaits where classical curve fitting-based approaches fail.
APA
Naish, J., Rodriguez, J., Zhang, J., Jones, B., Daddi, G., Orekhov, A., Royce, R., Paton, M., Choset, H., Ono, M. & Thakker, R.. (2024). Reinforcement learning-driven parametric curve fitting for snake robot gait design. Proceedings of the 6th Annual Learning for Dynamics & Control Conference, in Proceedings of Machine Learning Research 242:1715-1727 Available from https://proceedings.mlr.press/v242/naish24a.html.

Related Material