Neural Robot Dynamics

Jie Xu, Eric Heiden, Iretiayo Akinola, Dieter Fox, Miles Macklin, Yashraj Narang
Proceedings of The 9th Conference on Robot Learning, PMLR 305:3915-3935, 2025.

Abstract

Accurate and efficient simulation of modern robots remains challenging due to their high degrees of freedom and intricate mechanisms. Neural simulators have emerged as a promising alternative to traditional analytical simulators, capable of efficiently predicting complex dynamics and adapting to real-world data; however, existing neural simulators typically require application-specific training and fail to generalize to novel tasks and/or environments, primarily due to inadequate representations of the global state. In this work, we address the problem of learning generalizable neural simulators for robots that are structured as articulated rigid bodies. We propose *NeRD* (Neural Robot Dynamics), learned robot-specific dynamics models for predicting future states for articulated rigid bodies under contact constraints. *NeRD* uniquely replaces the low-level dynamics and contact solvers in an analytical simulator and employs a robot-centric and spatially-invariant simulation state representation. We integrate the learned *NeRD* models as an interchangeable backend solver within a state-of-the-art robotics simulator. We conduct extensive experiments to show that the *NeRD* simulators are stable and accurate over a thousand simulation steps; generalize across tasks and environment configurations; enable policy learning exclusively in a neural engine; and, unlike most classical simulators, can be fine-tuned from real-world data to bridge the gap between simulation and reality.

Cite this Paper

BibTeX
@InProceedings{pmlr-v305-xu25g, title = {Neural Robot Dynamics}, author = {Xu, Jie and Heiden, Eric and Akinola, Iretiayo and Fox, Dieter and Macklin, Miles and Narang, Yashraj}, booktitle = {Proceedings of The 9th Conference on Robot Learning}, pages = {3915--3935}, year = {2025}, editor = {Lim, Joseph and Song, Shuran and Park, Hae-Won}, volume = {305}, series = {Proceedings of Machine Learning Research}, month = {27--30 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v305/main/assets/xu25g/xu25g.pdf}, url = {https://proceedings.mlr.press/v305/xu25g.html}, abstract = {Accurate and efficient simulation of modern robots remains challenging due to their high degrees of freedom and intricate mechanisms. Neural simulators have emerged as a promising alternative to traditional analytical simulators, capable of efficiently predicting complex dynamics and adapting to real-world data; however, existing neural simulators typically require application-specific training and fail to generalize to novel tasks and/or environments, primarily due to inadequate representations of the global state. In this work, we address the problem of learning generalizable neural simulators for robots that are structured as articulated rigid bodies. We propose *NeRD* (Neural Robot Dynamics), learned robot-specific dynamics models for predicting future states for articulated rigid bodies under contact constraints. *NeRD* uniquely replaces the low-level dynamics and contact solvers in an analytical simulator and employs a robot-centric and spatially-invariant simulation state representation. We integrate the learned *NeRD* models as an interchangeable backend solver within a state-of-the-art robotics simulator. We conduct extensive experiments to show that the *NeRD* simulators are stable and accurate over a thousand simulation steps; generalize across tasks and environment configurations; enable policy learning exclusively in a neural engine; and, unlike most classical simulators, can be fine-tuned from real-world data to bridge the gap between simulation and reality.} }
Endnote
%0 Conference Paper %T Neural Robot Dynamics %A Jie Xu %A Eric Heiden %A Iretiayo Akinola %A Dieter Fox %A Miles Macklin %A Yashraj Narang %B Proceedings of The 9th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Joseph Lim %E Shuran Song %E Hae-Won Park %F pmlr-v305-xu25g %I PMLR %P 3915--3935 %U https://proceedings.mlr.press/v305/xu25g.html %V 305 %X Accurate and efficient simulation of modern robots remains challenging due to their high degrees of freedom and intricate mechanisms. Neural simulators have emerged as a promising alternative to traditional analytical simulators, capable of efficiently predicting complex dynamics and adapting to real-world data; however, existing neural simulators typically require application-specific training and fail to generalize to novel tasks and/or environments, primarily due to inadequate representations of the global state. In this work, we address the problem of learning generalizable neural simulators for robots that are structured as articulated rigid bodies. We propose *NeRD* (Neural Robot Dynamics), learned robot-specific dynamics models for predicting future states for articulated rigid bodies under contact constraints. *NeRD* uniquely replaces the low-level dynamics and contact solvers in an analytical simulator and employs a robot-centric and spatially-invariant simulation state representation. We integrate the learned *NeRD* models as an interchangeable backend solver within a state-of-the-art robotics simulator. We conduct extensive experiments to show that the *NeRD* simulators are stable and accurate over a thousand simulation steps; generalize across tasks and environment configurations; enable policy learning exclusively in a neural engine; and, unlike most classical simulators, can be fine-tuned from real-world data to bridge the gap between simulation and reality.
APA
Xu, J., Heiden, E., Akinola, I., Fox, D., Macklin, M. & Narang, Y.. (2025). Neural Robot Dynamics. Proceedings of The 9th Conference on Robot Learning, in Proceedings of Machine Learning Research 305:3915-3935 Available from https://proceedings.mlr.press/v305/xu25g.html.

Related Material