LARA: Latent Action Representation Alignment for Vision-Language-Action Models

Mengya Liu, Baoxiong Jia, Jiangyong Huang, Jingze Zhang, Siyuan Huang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:76855-76870, 2026.

Abstract

Visual-language-action (VLA) models enable robots to predict actions directly from observations and language instructions, but their performance depends on large-scale, high-quality data and is limited by the scarcity of real-world robot action datasets. To facilitate VLA model learning with abundant unlabeled human videos, Latent Action Models (LAM) learn latent action representations from visual dynamics to provide additional supervision for VLA learning. However, LAM and VLA are typically trained separately, leaving LAM ungrounded during VLA training and VLA models constrained by frozen LAM representations. To address these issues, we propose Latent Action Representation Alignment (LARA), a plug-and-play framework that jointly optimizes LAM and VLA via representation alignment. This enables reciprocal benefits where LAMs learn with action trajectories to avoid spurious visual changes, while VLAs are regularized by forward dynamics learned within LAMs to reduce hallucinations of functionally ineffective trajectories. We demonstrate LARA’s versatility and effectiveness for pre-training, post-training enhancement of pre-trained VLA models, and LAM refinement, achieving an average of 10%, 5%, and 15% improvement over 3 simulation and 1 meticulously designed real-world robotic manipulation benchmarks. The code is publicly available at https://github.com/lmy1001/LARA.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-liu26bl, title = {{LARA}: Latent Action Representation Alignment for Vision-Language-Action Models}, author = {Liu, Mengya and Jia, Baoxiong and Huang, Jiangyong and Zhang, Jingze and Huang, Siyuan}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {76855--76870}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/liu26bl/liu26bl.pdf}, url = {https://proceedings.mlr.press/v306/liu26bl.html}, abstract = {Visual-language-action (VLA) models enable robots to predict actions directly from observations and language instructions, but their performance depends on large-scale, high-quality data and is limited by the scarcity of real-world robot action datasets. To facilitate VLA model learning with abundant unlabeled human videos, Latent Action Models (LAM) learn latent action representations from visual dynamics to provide additional supervision for VLA learning. However, LAM and VLA are typically trained separately, leaving LAM ungrounded during VLA training and VLA models constrained by frozen LAM representations. To address these issues, we propose Latent Action Representation Alignment (LARA), a plug-and-play framework that jointly optimizes LAM and VLA via representation alignment. This enables reciprocal benefits where LAMs learn with action trajectories to avoid spurious visual changes, while VLAs are regularized by forward dynamics learned within LAMs to reduce hallucinations of functionally ineffective trajectories. We demonstrate LARA’s versatility and effectiveness for pre-training, post-training enhancement of pre-trained VLA models, and LAM refinement, achieving an average of 10%, 5%, and 15% improvement over 3 simulation and 1 meticulously designed real-world robotic manipulation benchmarks. The code is publicly available at https://github.com/lmy1001/LARA.} }
Endnote
%0 Conference Paper %T LARA: Latent Action Representation Alignment for Vision-Language-Action Models %A Mengya Liu %A Baoxiong Jia %A Jiangyong Huang %A Jingze Zhang %A Siyuan Huang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-liu26bl %I PMLR %P 76855--76870 %U https://proceedings.mlr.press/v306/liu26bl.html %V 306 %X Visual-language-action (VLA) models enable robots to predict actions directly from observations and language instructions, but their performance depends on large-scale, high-quality data and is limited by the scarcity of real-world robot action datasets. To facilitate VLA model learning with abundant unlabeled human videos, Latent Action Models (LAM) learn latent action representations from visual dynamics to provide additional supervision for VLA learning. However, LAM and VLA are typically trained separately, leaving LAM ungrounded during VLA training and VLA models constrained by frozen LAM representations. To address these issues, we propose Latent Action Representation Alignment (LARA), a plug-and-play framework that jointly optimizes LAM and VLA via representation alignment. This enables reciprocal benefits where LAMs learn with action trajectories to avoid spurious visual changes, while VLAs are regularized by forward dynamics learned within LAMs to reduce hallucinations of functionally ineffective trajectories. We demonstrate LARA’s versatility and effectiveness for pre-training, post-training enhancement of pre-trained VLA models, and LAM refinement, achieving an average of 10%, 5%, and 15% improvement over 3 simulation and 1 meticulously designed real-world robotic manipulation benchmarks. The code is publicly available at https://github.com/lmy1001/LARA.
APA
Liu, M., Jia, B., Huang, J., Zhang, J. & Huang, S.. (2026). LARA: Latent Action Representation Alignment for Vision-Language-Action Models. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:76855-76870 Available from https://proceedings.mlr.press/v306/liu26bl.html.

Related Material