Can VLMs Diagnose and Recover from VLA Manipulation Faults?

Bowen Yan, Jiahao Xiao, Kehui Liu, Jianbo Zhang, Zicheng Zhang, Qi Jia, Zhongjie Jia, Haoming Song, Chunyi Li, Bin Zhao, Guangtao Zhai
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:143967-143983, 2026.

Abstract

Existing VLA models frequently fail in robotic manipulation tasks, with poorly structured fault types that often require expert diagnosis. While VLMs offer strong explanatory capabilities, their effectiveness in assisting VLAs is limited by their unclear role in diagnostics and inadequate collaboration mechanisms. To address this, we introduce VLA-FixBench, a fault evaluation dataset that spans perception, planning, and control failures, and provides annotations for task stages, fault types, and spatiotemporal repair strategies. We further propose FaultEval, a static-to-dynamic-to-real evaluation framework that benchmarks 20 VLMs across multiple fault-related dimensions. Building on these insights, we design a VLM–VLA collaboration mechanism that localizes spatiotemporal deviations and rolls back task execution to enable targeted recovery. Experiments show that FaultEval reliably characterizes VLM-based closed-loop diagnosis and repair. The upper-bound analysis using human expert intervention shows that an idealized feedback loop can improve task success rates by 13% on LIBERO and 35% on real-world robots. Our code, benchmark, and project page will be publicly released at: https://kakigo.github.io/VLA-FixBench/

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-yan26r, title = {Can {VLM}s Diagnose and Recover from {VLA} Manipulation Faults?}, author = {Yan, Bowen and Xiao, Jiahao and Liu, Kehui and Zhang, Jianbo and Zhang, Zicheng and Jia, Qi and Jia, Zhongjie and Song, Haoming and Li, Chunyi and Zhao, Bin and Zhai, Guangtao}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {143967--143983}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/yan26r/yan26r.pdf}, url = {https://proceedings.mlr.press/v306/yan26r.html}, abstract = {Existing VLA models frequently fail in robotic manipulation tasks, with poorly structured fault types that often require expert diagnosis. While VLMs offer strong explanatory capabilities, their effectiveness in assisting VLAs is limited by their unclear role in diagnostics and inadequate collaboration mechanisms. To address this, we introduce VLA-FixBench, a fault evaluation dataset that spans perception, planning, and control failures, and provides annotations for task stages, fault types, and spatiotemporal repair strategies. We further propose FaultEval, a static-to-dynamic-to-real evaluation framework that benchmarks 20 VLMs across multiple fault-related dimensions. Building on these insights, we design a VLM–VLA collaboration mechanism that localizes spatiotemporal deviations and rolls back task execution to enable targeted recovery. Experiments show that FaultEval reliably characterizes VLM-based closed-loop diagnosis and repair. The upper-bound analysis using human expert intervention shows that an idealized feedback loop can improve task success rates by 13% on LIBERO and 35% on real-world robots. Our code, benchmark, and project page will be publicly released at: https://kakigo.github.io/VLA-FixBench/} }
Endnote
%0 Conference Paper %T Can VLMs Diagnose and Recover from VLA Manipulation Faults? %A Bowen Yan %A Jiahao Xiao %A Kehui Liu %A Jianbo Zhang %A Zicheng Zhang %A Qi Jia %A Zhongjie Jia %A Haoming Song %A Chunyi Li %A Bin Zhao %A Guangtao Zhai %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-yan26r %I PMLR %P 143967--143983 %U https://proceedings.mlr.press/v306/yan26r.html %V 306 %X Existing VLA models frequently fail in robotic manipulation tasks, with poorly structured fault types that often require expert diagnosis. While VLMs offer strong explanatory capabilities, their effectiveness in assisting VLAs is limited by their unclear role in diagnostics and inadequate collaboration mechanisms. To address this, we introduce VLA-FixBench, a fault evaluation dataset that spans perception, planning, and control failures, and provides annotations for task stages, fault types, and spatiotemporal repair strategies. We further propose FaultEval, a static-to-dynamic-to-real evaluation framework that benchmarks 20 VLMs across multiple fault-related dimensions. Building on these insights, we design a VLM–VLA collaboration mechanism that localizes spatiotemporal deviations and rolls back task execution to enable targeted recovery. Experiments show that FaultEval reliably characterizes VLM-based closed-loop diagnosis and repair. The upper-bound analysis using human expert intervention shows that an idealized feedback loop can improve task success rates by 13% on LIBERO and 35% on real-world robots. Our code, benchmark, and project page will be publicly released at: https://kakigo.github.io/VLA-FixBench/
APA
Yan, B., Xiao, J., Liu, K., Zhang, J., Zhang, Z., Jia, Q., Jia, Z., Song, H., Li, C., Zhao, B. & Zhai, G.. (2026). Can VLMs Diagnose and Recover from VLA Manipulation Faults?. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:143967-143983 Available from https://proceedings.mlr.press/v306/yan26r.html.

Related Material