SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-Body Manipulation

Mu Huang, Hui Wang, Kerui Ren, Linning Xu, Yunsong Zhou, Mulin Yu, Bo Dai, Jiangmiao Pang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:47834-47850, 2026.

Abstract

Simulating deformable objects under rich interactions remains a fundamental challenge for real-to-sim robot manipulation, with dynamics jointly driven by environmental effects and robot actions. Existing simulators rely on predefined physics or data-driven dynamics without robot-conditioned control, limiting accuracy, stability, and generalization. This paper presents SoMA, a 3D Gaussian Splat simulator for soft-body manipulation. SoMA couples deformable dynamics, environmental forces, and robot joint actions in a unified latent neural space for end-to-end real-to-sim simulation. Modeling interactions over learned Gaussian splats enables controllable, stable long-horizon manipulation and generalization beyond observed trajectories without predefined physical models. SoMA improves resimulation accuracy and generalization on real-world robot manipulation by 20%, enabling stable simulation of complex tasks such as long-horizon cloth folding. Project Page: city-super.github.io/SoMA

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-huang26bh, title = {{S}o{MA}: A Real-to-Sim Neural Simulator for Robotic Soft-Body Manipulation}, author = {Huang, Mu and Wang, Hui and Ren, Kerui and Xu, Linning and Zhou, Yunsong and Yu, Mulin and Dai, Bo and Pang, Jiangmiao}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {47834--47850}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/huang26bh/huang26bh.pdf}, url = {https://proceedings.mlr.press/v306/huang26bh.html}, abstract = {Simulating deformable objects under rich interactions remains a fundamental challenge for real-to-sim robot manipulation, with dynamics jointly driven by environmental effects and robot actions. Existing simulators rely on predefined physics or data-driven dynamics without robot-conditioned control, limiting accuracy, stability, and generalization. This paper presents SoMA, a 3D Gaussian Splat simulator for soft-body manipulation. SoMA couples deformable dynamics, environmental forces, and robot joint actions in a unified latent neural space for end-to-end real-to-sim simulation. Modeling interactions over learned Gaussian splats enables controllable, stable long-horizon manipulation and generalization beyond observed trajectories without predefined physical models. SoMA improves resimulation accuracy and generalization on real-world robot manipulation by 20%, enabling stable simulation of complex tasks such as long-horizon cloth folding. Project Page: city-super.github.io/SoMA} }
Endnote
%0 Conference Paper %T SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-Body Manipulation %A Mu Huang %A Hui Wang %A Kerui Ren %A Linning Xu %A Yunsong Zhou %A Mulin Yu %A Bo Dai %A Jiangmiao Pang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-huang26bh %I PMLR %P 47834--47850 %U https://proceedings.mlr.press/v306/huang26bh.html %V 306 %X Simulating deformable objects under rich interactions remains a fundamental challenge for real-to-sim robot manipulation, with dynamics jointly driven by environmental effects and robot actions. Existing simulators rely on predefined physics or data-driven dynamics without robot-conditioned control, limiting accuracy, stability, and generalization. This paper presents SoMA, a 3D Gaussian Splat simulator for soft-body manipulation. SoMA couples deformable dynamics, environmental forces, and robot joint actions in a unified latent neural space for end-to-end real-to-sim simulation. Modeling interactions over learned Gaussian splats enables controllable, stable long-horizon manipulation and generalization beyond observed trajectories without predefined physical models. SoMA improves resimulation accuracy and generalization on real-world robot manipulation by 20%, enabling stable simulation of complex tasks such as long-horizon cloth folding. Project Page: city-super.github.io/SoMA
APA
Huang, M., Wang, H., Ren, K., Xu, L., Zhou, Y., Yu, M., Dai, B. & Pang, J.. (2026). SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-Body Manipulation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:47834-47850 Available from https://proceedings.mlr.press/v306/huang26bh.html.

Related Material