Motion Dynamics Learning for Few-Shot Embodied Adaptation

Sibo He, Weiying Xie, Daixun Li, Junhao Zhong, Jiayun Tian, Yunke Wang, Leyuan Fang, Gang He, Yunsong Li
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:42022-42046, 2026.

Abstract

Vision-Language-Action (VLA) models have shown strong potential for robotic manipulation, yet adapting pretrained models to novel tasks typically relies on substantial task-specific demonstrations, limiting scalability. Current VLA methods mostly focus on action imitation, which ignores the richer structure contained in trajectories. In contrast, motion dynamics governing how actions evolve over time are more informative and transferable, making them better suited for few-shot adaptation. Motivated by this idea, we propose DynVLA, a few-shot adaptation system that reformulates VLA learning from action imitation to trajectory-level motion dynamics modeling. Specifically, we propose Motion Dynamics Mechanism (MDM), which distills latent embeddings from trajectories via flow-matching inversion, yielding compact representations that capture dynamics. We further design Dynamics-Constrained Modeling (DCM). DCM projects these inferred representations onto a Dynamics Bank, which stores prior motion knowledge pretrained from diverse demonstrations. By grounding action generation in these learned priors, the system enables interpolating between existing action paradigms to represent novel dynamics modes. Experiments on 13 real-world tasks demonstrate that DynVLA outperforms existing SOTA systems by 15.8% in average success rate with around 20 demonstrations, highlighting its adaptation capabilities in real-world scenes. Our code is available at https://github.com/trantor2nd/Motion-DynVLA.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-he26ao, title = {Motion Dynamics Learning for Few-Shot Embodied Adaptation}, author = {He, Sibo and Xie, Weiying and Li, Daixun and Zhong, Junhao and Tian, Jiayun and Wang, Yunke and Fang, Leyuan and He, Gang and Li, Yunsong}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {42022--42046}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/he26ao/he26ao.pdf}, url = {https://proceedings.mlr.press/v306/he26ao.html}, abstract = {Vision-Language-Action (VLA) models have shown strong potential for robotic manipulation, yet adapting pretrained models to novel tasks typically relies on substantial task-specific demonstrations, limiting scalability. Current VLA methods mostly focus on action imitation, which ignores the richer structure contained in trajectories. In contrast, motion dynamics governing how actions evolve over time are more informative and transferable, making them better suited for few-shot adaptation. Motivated by this idea, we propose DynVLA, a few-shot adaptation system that reformulates VLA learning from action imitation to trajectory-level motion dynamics modeling. Specifically, we propose Motion Dynamics Mechanism (MDM), which distills latent embeddings from trajectories via flow-matching inversion, yielding compact representations that capture dynamics. We further design Dynamics-Constrained Modeling (DCM). DCM projects these inferred representations onto a Dynamics Bank, which stores prior motion knowledge pretrained from diverse demonstrations. By grounding action generation in these learned priors, the system enables interpolating between existing action paradigms to represent novel dynamics modes. Experiments on 13 real-world tasks demonstrate that DynVLA outperforms existing SOTA systems by 15.8% in average success rate with around 20 demonstrations, highlighting its adaptation capabilities in real-world scenes. Our code is available at https://github.com/trantor2nd/Motion-DynVLA.} }
Endnote
%0 Conference Paper %T Motion Dynamics Learning for Few-Shot Embodied Adaptation %A Sibo He %A Weiying Xie %A Daixun Li %A Junhao Zhong %A Jiayun Tian %A Yunke Wang %A Leyuan Fang %A Gang He %A Yunsong Li %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-he26ao %I PMLR %P 42022--42046 %U https://proceedings.mlr.press/v306/he26ao.html %V 306 %X Vision-Language-Action (VLA) models have shown strong potential for robotic manipulation, yet adapting pretrained models to novel tasks typically relies on substantial task-specific demonstrations, limiting scalability. Current VLA methods mostly focus on action imitation, which ignores the richer structure contained in trajectories. In contrast, motion dynamics governing how actions evolve over time are more informative and transferable, making them better suited for few-shot adaptation. Motivated by this idea, we propose DynVLA, a few-shot adaptation system that reformulates VLA learning from action imitation to trajectory-level motion dynamics modeling. Specifically, we propose Motion Dynamics Mechanism (MDM), which distills latent embeddings from trajectories via flow-matching inversion, yielding compact representations that capture dynamics. We further design Dynamics-Constrained Modeling (DCM). DCM projects these inferred representations onto a Dynamics Bank, which stores prior motion knowledge pretrained from diverse demonstrations. By grounding action generation in these learned priors, the system enables interpolating between existing action paradigms to represent novel dynamics modes. Experiments on 13 real-world tasks demonstrate that DynVLA outperforms existing SOTA systems by 15.8% in average success rate with around 20 demonstrations, highlighting its adaptation capabilities in real-world scenes. Our code is available at https://github.com/trantor2nd/Motion-DynVLA.
APA
He, S., Xie, W., Li, D., Zhong, J., Tian, J., Wang, Y., Fang, L., He, G. & Li, Y.. (2026). Motion Dynamics Learning for Few-Shot Embodied Adaptation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:42022-42046 Available from https://proceedings.mlr.press/v306/he26ao.html.

Related Material