AdaNav: Adaptive Reasoning with Uncertainty for Vision-Language Navigation

Xin Ding, Jianyu Wei, Yifan Yang, Shiqi Jiang, Qianxi Zhang, Hao Wu, Fucheng Jia, Liang Mi, Yuxuan Yan, Weijun Wang, Yunxin Liu, Zhibo Chen, Ting Cao
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:25205-25221, 2026.

Abstract

Vision-Language Navigation (VLN) requires agents to follow natural language instructions by grounding them in sequential visual observations over long horizons. Explicit reasoning could enhance temporal consistency and perception–action alignment, but reasoning at fixed steps often leads to suboptimal performance and unnecessary computation. To address this, we propose AdaNav, an uncertainty-based adaptive reasoning framework for VLN. At its core is the Uncertainty-Adaptive Reasoning Block (UAR), a lightweight plugin that dynamically triggers reasoning. We introduce Action Entropy as a policy prior for UAR and progressively refine it through a Heuristics-to-RL training method, enabling agents to learn difficulty-aware reasoning policies under the strict data limitations of embodied tasks. Results show that with only 6K training samples, AdaNav achieves substantial gains over closed-source models trained on million-scale data, improving success rate by 20% on R2R val-unseen, 11.7% on RxR-CE, and 11.4% in real-world scenes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-ding26r, title = {{A}da{N}av: Adaptive Reasoning with Uncertainty for Vision-Language Navigation}, author = {Ding, Xin and Wei, Jianyu and Yang, Yifan and Jiang, Shiqi and Zhang, Qianxi and Wu, Hao and Jia, Fucheng and Mi, Liang and Yan, Yuxuan and Wang, Weijun and Liu, Yunxin and Chen, Zhibo and Cao, Ting}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {25205--25221}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/ding26r/ding26r.pdf}, url = {https://proceedings.mlr.press/v306/ding26r.html}, abstract = {Vision-Language Navigation (VLN) requires agents to follow natural language instructions by grounding them in sequential visual observations over long horizons. Explicit reasoning could enhance temporal consistency and perception–action alignment, but reasoning at fixed steps often leads to suboptimal performance and unnecessary computation. To address this, we propose AdaNav, an uncertainty-based adaptive reasoning framework for VLN. At its core is the Uncertainty-Adaptive Reasoning Block (UAR), a lightweight plugin that dynamically triggers reasoning. We introduce Action Entropy as a policy prior for UAR and progressively refine it through a Heuristics-to-RL training method, enabling agents to learn difficulty-aware reasoning policies under the strict data limitations of embodied tasks. Results show that with only 6K training samples, AdaNav achieves substantial gains over closed-source models trained on million-scale data, improving success rate by 20% on R2R val-unseen, 11.7% on RxR-CE, and 11.4% in real-world scenes.} }
Endnote
%0 Conference Paper %T AdaNav: Adaptive Reasoning with Uncertainty for Vision-Language Navigation %A Xin Ding %A Jianyu Wei %A Yifan Yang %A Shiqi Jiang %A Qianxi Zhang %A Hao Wu %A Fucheng Jia %A Liang Mi %A Yuxuan Yan %A Weijun Wang %A Yunxin Liu %A Zhibo Chen %A Ting Cao %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-ding26r %I PMLR %P 25205--25221 %U https://proceedings.mlr.press/v306/ding26r.html %V 306 %X Vision-Language Navigation (VLN) requires agents to follow natural language instructions by grounding them in sequential visual observations over long horizons. Explicit reasoning could enhance temporal consistency and perception–action alignment, but reasoning at fixed steps often leads to suboptimal performance and unnecessary computation. To address this, we propose AdaNav, an uncertainty-based adaptive reasoning framework for VLN. At its core is the Uncertainty-Adaptive Reasoning Block (UAR), a lightweight plugin that dynamically triggers reasoning. We introduce Action Entropy as a policy prior for UAR and progressively refine it through a Heuristics-to-RL training method, enabling agents to learn difficulty-aware reasoning policies under the strict data limitations of embodied tasks. Results show that with only 6K training samples, AdaNav achieves substantial gains over closed-source models trained on million-scale data, improving success rate by 20% on R2R val-unseen, 11.7% on RxR-CE, and 11.4% in real-world scenes.
APA
Ding, X., Wei, J., Yang, Y., Jiang, S., Zhang, Q., Wu, H., Jia, F., Mi, L., Yan, Y., Wang, W., Liu, Y., Chen, Z. & Cao, T.. (2026). AdaNav: Adaptive Reasoning with Uncertainty for Vision-Language Navigation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:25205-25221 Available from https://proceedings.mlr.press/v306/ding26r.html.

Related Material