Learning Deployable Locomotion Control via Differentiable Simulation

Clemens Schwarke, Victor Klemm, Joshua Bagajo, Jean Pierre Sleiman, Ignat Georgiev, Jesus Tordesillas Torres, Marco Hutter
Proceedings of The 9th Conference on Robot Learning, PMLR 305:3665-3684, 2025.

Abstract

Differentiable simulators promise to improve sample efficiency in robot learning by providing analytic gradients of the system dynamics. Yet, their application to contact-rich tasks like locomotion is complicated by the inherently non-smooth nature of contact, impeding effective gradient-based optimization. Existing works thus often rely on soft contact models that provide smooth gradients but lack physical accuracy, constraining results to simulation. To address this limitation, we propose a differentiable contact model designed to provide informative gradients while maintaining high physical fidelity. We demonstrate the efficacy of our approach by training a quadrupedal locomotion policy within our differentiable simulator leveraging analytic gradients and successfully transferring the learned policy zero-shot to the real world. To the best of our knowledge, this represents the first successful sim-to-real transfer of a legged locomotion policy learned entirely within a differentiable simulator, establishing the feasibility of using differentiable simulation for real-world locomotion control.

Cite this Paper

BibTeX
@InProceedings{pmlr-v305-schwarke25a, title = {Learning Deployable Locomotion Control via Differentiable Simulation}, author = {Schwarke, Clemens and Klemm, Victor and Bagajo, Joshua and Sleiman, Jean Pierre and Georgiev, Ignat and Torres, Jesus Tordesillas and Hutter, Marco}, booktitle = {Proceedings of The 9th Conference on Robot Learning}, pages = {3665--3684}, year = {2025}, editor = {Lim, Joseph and Song, Shuran and Park, Hae-Won}, volume = {305}, series = {Proceedings of Machine Learning Research}, month = {27--30 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v305/main/assets/schwarke25a/schwarke25a.pdf}, url = {https://proceedings.mlr.press/v305/schwarke25a.html}, abstract = {Differentiable simulators promise to improve sample efficiency in robot learning by providing analytic gradients of the system dynamics. Yet, their application to contact-rich tasks like locomotion is complicated by the inherently non-smooth nature of contact, impeding effective gradient-based optimization. Existing works thus often rely on soft contact models that provide smooth gradients but lack physical accuracy, constraining results to simulation. To address this limitation, we propose a differentiable contact model designed to provide informative gradients while maintaining high physical fidelity. We demonstrate the efficacy of our approach by training a quadrupedal locomotion policy within our differentiable simulator leveraging analytic gradients and successfully transferring the learned policy zero-shot to the real world. To the best of our knowledge, this represents the first successful sim-to-real transfer of a legged locomotion policy learned entirely within a differentiable simulator, establishing the feasibility of using differentiable simulation for real-world locomotion control.} }
Endnote
%0 Conference Paper %T Learning Deployable Locomotion Control via Differentiable Simulation %A Clemens Schwarke %A Victor Klemm %A Joshua Bagajo %A Jean Pierre Sleiman %A Ignat Georgiev %A Jesus Tordesillas Torres %A Marco Hutter %B Proceedings of The 9th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Joseph Lim %E Shuran Song %E Hae-Won Park %F pmlr-v305-schwarke25a %I PMLR %P 3665--3684 %U https://proceedings.mlr.press/v305/schwarke25a.html %V 305 %X Differentiable simulators promise to improve sample efficiency in robot learning by providing analytic gradients of the system dynamics. Yet, their application to contact-rich tasks like locomotion is complicated by the inherently non-smooth nature of contact, impeding effective gradient-based optimization. Existing works thus often rely on soft contact models that provide smooth gradients but lack physical accuracy, constraining results to simulation. To address this limitation, we propose a differentiable contact model designed to provide informative gradients while maintaining high physical fidelity. We demonstrate the efficacy of our approach by training a quadrupedal locomotion policy within our differentiable simulator leveraging analytic gradients and successfully transferring the learned policy zero-shot to the real world. To the best of our knowledge, this represents the first successful sim-to-real transfer of a legged locomotion policy learned entirely within a differentiable simulator, establishing the feasibility of using differentiable simulation for real-world locomotion control.
APA
Schwarke, C., Klemm, V., Bagajo, J., Sleiman, J.P., Georgiev, I., Torres, J.T. & Hutter, M.. (2025). Learning Deployable Locomotion Control via Differentiable Simulation. Proceedings of The 9th Conference on Robot Learning, in Proceedings of Machine Learning Research 305:3665-3684 Available from https://proceedings.mlr.press/v305/schwarke25a.html.

Related Material