Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving

Mingyi Wang, Jingke Wang, Tengju Ye, Kaicheng Yu
Proceedings of The 9th Conference on Robot Learning, PMLR 305:4657-4683, 2025.

Abstract

Recent breakthroughs in large language models (LLMs) have not only advanced natural language processing but also inspired their application in domains with structurally similar problems—most notably, autonomous driving motion generation. Both domains involve autoregressive sequence modeling, token-based representations, and context-aware decision making, making the transfer of LLM components a natural and increasingly common practice. However, despite promising early attempts, a systematic understanding of which LLM modules are truly transferable remains lacking. In this paper, we present a comprehensive evaluation of five key LLM modules—tokenizer design, positional embedding, pre-training paradigms, post-training strategies, and test-time computation—within the context of motion generation for autonomous driving. Through extensive experiments on the Waymo Sim Agents benchmark, we demonstrate that, when appropriately adapted, these modules can significantly improve performance for autonomous driving motion generation. In addition, we identify which techniques can be effectively transferred, analyze the potential reasons for the failure of others, and discuss the specific adaptations needed for autonomous driving scenarios. We evaluate our method on the Sim Agents task and achieve competitive results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v305-wang25g, title = {Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving}, author = {Wang, Mingyi and Wang, Jingke and Ye, Tengju and Yu, Kaicheng}, booktitle = {Proceedings of The 9th Conference on Robot Learning}, pages = {4657--4683}, year = {2025}, editor = {Lim, Joseph and Song, Shuran and Park, Hae-Won}, volume = {305}, series = {Proceedings of Machine Learning Research}, month = {27--30 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v305/main/assets/wang25g/wang25g.pdf}, url = {https://proceedings.mlr.press/v305/wang25g.html}, abstract = {Recent breakthroughs in large language models (LLMs) have not only advanced natural language processing but also inspired their application in domains with structurally similar problems—most notably, autonomous driving motion generation. Both domains involve autoregressive sequence modeling, token-based representations, and context-aware decision making, making the transfer of LLM components a natural and increasingly common practice. However, despite promising early attempts, a systematic understanding of which LLM modules are truly transferable remains lacking. In this paper, we present a comprehensive evaluation of five key LLM modules—tokenizer design, positional embedding, pre-training paradigms, post-training strategies, and test-time computation—within the context of motion generation for autonomous driving. Through extensive experiments on the Waymo Sim Agents benchmark, we demonstrate that, when appropriately adapted, these modules can significantly improve performance for autonomous driving motion generation. In addition, we identify which techniques can be effectively transferred, analyze the potential reasons for the failure of others, and discuss the specific adaptations needed for autonomous driving scenarios. We evaluate our method on the Sim Agents task and achieve competitive results.} }
Endnote
%0 Conference Paper %T Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving %A Mingyi Wang %A Jingke Wang %A Tengju Ye %A Kaicheng Yu %B Proceedings of The 9th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Joseph Lim %E Shuran Song %E Hae-Won Park %F pmlr-v305-wang25g %I PMLR %P 4657--4683 %U https://proceedings.mlr.press/v305/wang25g.html %V 305 %X Recent breakthroughs in large language models (LLMs) have not only advanced natural language processing but also inspired their application in domains with structurally similar problems—most notably, autonomous driving motion generation. Both domains involve autoregressive sequence modeling, token-based representations, and context-aware decision making, making the transfer of LLM components a natural and increasingly common practice. However, despite promising early attempts, a systematic understanding of which LLM modules are truly transferable remains lacking. In this paper, we present a comprehensive evaluation of five key LLM modules—tokenizer design, positional embedding, pre-training paradigms, post-training strategies, and test-time computation—within the context of motion generation for autonomous driving. Through extensive experiments on the Waymo Sim Agents benchmark, we demonstrate that, when appropriately adapted, these modules can significantly improve performance for autonomous driving motion generation. In addition, we identify which techniques can be effectively transferred, analyze the potential reasons for the failure of others, and discuss the specific adaptations needed for autonomous driving scenarios. We evaluate our method on the Sim Agents task and achieve competitive results.
APA
Wang, M., Wang, J., Ye, T. & Yu, K.. (2025). Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving. Proceedings of The 9th Conference on Robot Learning, in Proceedings of Machine Learning Research 305:4657-4683 Available from https://proceedings.mlr.press/v305/wang25g.html.

Related Material