DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving

Shuyao Shang, Bing Zhan, Yunfei Yan, Yuqi Wang, Yingyan Li, Yasong An, Xiaoman Wang, Jierui Liu, Lu Hou, Lue Fan, Zhaoxiang Zhang, Tieniu Tan
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:110028-110045, 2026.

Abstract

We propose DynVLA, a driving VLA model that introduces a new CoT paradigm termed Dynamics CoT. DynVLA forecasts compact world dynamics before action generation, enabling more informed and physically grounded decision-making. To obtain compact dynamics representations, DynVLA introduces a Dynamics Tokenizer that compresses future evolution into a small set of dynamics tokens. Considering the rich environment dynamics in interaction-intensive driving scenarios, DynVLA decouples ego-centric and environment-centric dynamics, yielding more accurate world dynamics modeling. We then train DynVLA to generate dynamics tokens before actions through SFT and RFT, improving decision quality while maintaining latency-efficient inference. Compared to Textual CoT, which lacks fine-grained spatiotemporal understanding, and Visual CoT, which introduces substantial redundancy due to dense image prediction, Dynamics CoT captures the evolution of the world in a compact, interpretable, and efficient form. Extensive experiments on NAVSIM, Bench2Drive, and a large-scale in-house dataset demonstrate that DynVLA consistently outperforms Textual CoT and Visual CoT methods, validating the effectiveness and practical value of Dynamics CoT.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-shang26f, title = {{D}yn{VLA}: Learning World Dynamics for Action Reasoning in Autonomous Driving}, author = {Shang, Shuyao and Zhan, Bing and Yan, Yunfei and Wang, Yuqi and Li, Yingyan and An, Yasong and Wang, Xiaoman and Liu, Jierui and Hou, Lu and Fan, Lue and Zhang, Zhaoxiang and Tan, Tieniu}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {110028--110045}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/shang26f/shang26f.pdf}, url = {https://proceedings.mlr.press/v306/shang26f.html}, abstract = {We propose DynVLA, a driving VLA model that introduces a new CoT paradigm termed Dynamics CoT. DynVLA forecasts compact world dynamics before action generation, enabling more informed and physically grounded decision-making. To obtain compact dynamics representations, DynVLA introduces a Dynamics Tokenizer that compresses future evolution into a small set of dynamics tokens. Considering the rich environment dynamics in interaction-intensive driving scenarios, DynVLA decouples ego-centric and environment-centric dynamics, yielding more accurate world dynamics modeling. We then train DynVLA to generate dynamics tokens before actions through SFT and RFT, improving decision quality while maintaining latency-efficient inference. Compared to Textual CoT, which lacks fine-grained spatiotemporal understanding, and Visual CoT, which introduces substantial redundancy due to dense image prediction, Dynamics CoT captures the evolution of the world in a compact, interpretable, and efficient form. Extensive experiments on NAVSIM, Bench2Drive, and a large-scale in-house dataset demonstrate that DynVLA consistently outperforms Textual CoT and Visual CoT methods, validating the effectiveness and practical value of Dynamics CoT.} }
Endnote
%0 Conference Paper %T DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving %A Shuyao Shang %A Bing Zhan %A Yunfei Yan %A Yuqi Wang %A Yingyan Li %A Yasong An %A Xiaoman Wang %A Jierui Liu %A Lu Hou %A Lue Fan %A Zhaoxiang Zhang %A Tieniu Tan %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-shang26f %I PMLR %P 110028--110045 %U https://proceedings.mlr.press/v306/shang26f.html %V 306 %X We propose DynVLA, a driving VLA model that introduces a new CoT paradigm termed Dynamics CoT. DynVLA forecasts compact world dynamics before action generation, enabling more informed and physically grounded decision-making. To obtain compact dynamics representations, DynVLA introduces a Dynamics Tokenizer that compresses future evolution into a small set of dynamics tokens. Considering the rich environment dynamics in interaction-intensive driving scenarios, DynVLA decouples ego-centric and environment-centric dynamics, yielding more accurate world dynamics modeling. We then train DynVLA to generate dynamics tokens before actions through SFT and RFT, improving decision quality while maintaining latency-efficient inference. Compared to Textual CoT, which lacks fine-grained spatiotemporal understanding, and Visual CoT, which introduces substantial redundancy due to dense image prediction, Dynamics CoT captures the evolution of the world in a compact, interpretable, and efficient form. Extensive experiments on NAVSIM, Bench2Drive, and a large-scale in-house dataset demonstrate that DynVLA consistently outperforms Textual CoT and Visual CoT methods, validating the effectiveness and practical value of Dynamics CoT.
APA
Shang, S., Zhan, B., Yan, Y., Wang, Y., Li, Y., An, Y., Wang, X., Liu, J., Hou, L., Fan, L., Zhang, Z. & Tan, T.. (2026). DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:110028-110045 Available from https://proceedings.mlr.press/v306/shang26f.html.

Related Material