BEAR: Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis

Yu Qi, Haibo Zhao, Ziyu Guo, Siyuan Ma, Ziyan Chen, Yaokun Han, Renrui Zhang, Zitiantao Lin, Yizhe Zhu, Shiji Xin, Yijian Huang, Boce Hu, Kai Cheng, Jiayi Zhang, Peiheng Wang, Jiazheng Liu, Wenqing Wang, Yiran Qin, Haojie Huang, Lawson L.S. Wong
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:100575-100674, 2026.

Abstract

Understanding the capability bottlenecks of embodied multimodal large language models (MLLMs) is crucial for improvement. However, existing embodied benchmarks fail to provide actionable insights because they focus on task-level evaluation rather than discovering capability bottlenecks. To address this, we introduce BEAR, where we divide embodied tasks into 14 atomic skills for skill-level evaluation. BEAR comprises 4,469 interleaved image–video–text entries across 14 skills in 6 categories, ranging from low-level perception to high-level planning. We evaluate 20 MLLMs on BEAR under a hierarchical skill-level diagnosis framework and discover that (1) perceptual capabilities are major bottlenecks behind reasoning failures, and (2) models fail due to unstable spatiotemporal modeling which remain unexposed in previous benchmarks. Furthermore, building on these insights, we propose BEAR-Agent, a multimodal conversable agent that augments MLLMs with visual and spatial tools. It substantially enhances MLLMs’ performance across skills, yielding a relative improvement of 17.5% on GPT-5 on BEAR and outperform baselines by a large margin in both simulation and real-robot experiments across models. We provide our project website at https://bear-official66.github.io/.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-qi26p, title = {{BEAR}: Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis}, author = {Qi, Yu and Zhao, Haibo and Guo, Ziyu and Ma, Siyuan and Chen, Ziyan and Han, Yaokun and Zhang, Renrui and Lin, Zitiantao and Zhu, Yizhe and Xin, Shiji and Huang, Yijian and Hu, Boce and Cheng, Kai and Zhang, Jiayi and Wang, Peiheng and Liu, Jiazheng and Wang, Wenqing and Qin, Yiran and Huang, Haojie and Wong, Lawson L.S.}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {100575--100674}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/qi26p/qi26p.pdf}, url = {https://proceedings.mlr.press/v306/qi26p.html}, abstract = {Understanding the capability bottlenecks of embodied multimodal large language models (MLLMs) is crucial for improvement. However, existing embodied benchmarks fail to provide actionable insights because they focus on task-level evaluation rather than discovering capability bottlenecks. To address this, we introduce BEAR, where we divide embodied tasks into 14 atomic skills for skill-level evaluation. BEAR comprises 4,469 interleaved image–video–text entries across 14 skills in 6 categories, ranging from low-level perception to high-level planning. We evaluate 20 MLLMs on BEAR under a hierarchical skill-level diagnosis framework and discover that (1) perceptual capabilities are major bottlenecks behind reasoning failures, and (2) models fail due to unstable spatiotemporal modeling which remain unexposed in previous benchmarks. Furthermore, building on these insights, we propose BEAR-Agent, a multimodal conversable agent that augments MLLMs with visual and spatial tools. It substantially enhances MLLMs’ performance across skills, yielding a relative improvement of 17.5% on GPT-5 on BEAR and outperform baselines by a large margin in both simulation and real-robot experiments across models. We provide our project website at https://bear-official66.github.io/.} }
Endnote
%0 Conference Paper %T BEAR: Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis %A Yu Qi %A Haibo Zhao %A Ziyu Guo %A Siyuan Ma %A Ziyan Chen %A Yaokun Han %A Renrui Zhang %A Zitiantao Lin %A Yizhe Zhu %A Shiji Xin %A Yijian Huang %A Boce Hu %A Kai Cheng %A Jiayi Zhang %A Peiheng Wang %A Jiazheng Liu %A Wenqing Wang %A Yiran Qin %A Haojie Huang %A Lawson L.S. Wong %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-qi26p %I PMLR %P 100575--100674 %U https://proceedings.mlr.press/v306/qi26p.html %V 306 %X Understanding the capability bottlenecks of embodied multimodal large language models (MLLMs) is crucial for improvement. However, existing embodied benchmarks fail to provide actionable insights because they focus on task-level evaluation rather than discovering capability bottlenecks. To address this, we introduce BEAR, where we divide embodied tasks into 14 atomic skills for skill-level evaluation. BEAR comprises 4,469 interleaved image–video–text entries across 14 skills in 6 categories, ranging from low-level perception to high-level planning. We evaluate 20 MLLMs on BEAR under a hierarchical skill-level diagnosis framework and discover that (1) perceptual capabilities are major bottlenecks behind reasoning failures, and (2) models fail due to unstable spatiotemporal modeling which remain unexposed in previous benchmarks. Furthermore, building on these insights, we propose BEAR-Agent, a multimodal conversable agent that augments MLLMs with visual and spatial tools. It substantially enhances MLLMs’ performance across skills, yielding a relative improvement of 17.5% on GPT-5 on BEAR and outperform baselines by a large margin in both simulation and real-robot experiments across models. We provide our project website at https://bear-official66.github.io/.
APA
Qi, Y., Zhao, H., Guo, Z., Ma, S., Chen, Z., Han, Y., Zhang, R., Lin, Z., Zhu, Y., Xin, S., Huang, Y., Hu, B., Cheng, K., Zhang, J., Wang, P., Liu, J., Wang, W., Qin, Y., Huang, H. & Wong, L.L.. (2026). BEAR: Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:100575-100674 Available from https://proceedings.mlr.press/v306/qi26p.html.

Related Material