Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence

Jiahao Meng, Xiangtai Li, Haochen Wang, Tan Yue, Tao Zhang, Lingdong Kong, Yunhai Tong, Anran Wang, Zhiyang Teng, Yujing Wang, Zhuochen Wang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:87958-87979, 2026.

Abstract

Most video reasoning models only generate textual reasoning traces without indicating when and where key evidence appears. Recent models such as OpenAI-o3 have sparked wide interest in evidence-centered reasoning for images, yet extending this ability to videos is more challenging due to the need for joint temporal tracking and spatial localization across dynamic scenes. We introduce Open-o3-Video, a non-agent framework that integrates explicit spatio-temporal evidence into video reasoning by highlighting key timestamps, objects, and bounding boxes, making the reasoning process traceable and verifiable. To enable this capability, we first construct high-quality datasets STGR that provide unified spatio-temporal supervision, which is absent in existing resources. We further adopt a cold-start reinforcement learning strategy with specially designed rewards that jointly encourage answer accuracy, temporal alignment, and spatial precision. On the V-STAR benchmark, Open-o3-Video achieves state-of-the-art performance, improving mAM by 14.4% and mLGM by 24.2% over the Qwen2.5-VL baseline, and shows consistent gains across a range of video understanding benchmarks. Beyond accuracy, the grounded reasoning traces produced by Open-o3-Video support confidence-aware test-time scaling, improving answer reliability. The code, model and datasets are publicly available at https://marinero4972.github.io/projects/Open-o3-Video/.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-meng26e, title = {Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence}, author = {Meng, Jiahao and Li, Xiangtai and Wang, Haochen and Yue, Tan and Zhang, Tao and Kong, Lingdong and Tong, Yunhai and Wang, Anran and Teng, Zhiyang and Wang, Yujing and Wang, Zhuochen}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {87958--87979}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/meng26e/meng26e.pdf}, url = {https://proceedings.mlr.press/v306/meng26e.html}, abstract = {Most video reasoning models only generate textual reasoning traces without indicating when and where key evidence appears. Recent models such as OpenAI-o3 have sparked wide interest in evidence-centered reasoning for images, yet extending this ability to videos is more challenging due to the need for joint temporal tracking and spatial localization across dynamic scenes. We introduce Open-o3-Video, a non-agent framework that integrates explicit spatio-temporal evidence into video reasoning by highlighting key timestamps, objects, and bounding boxes, making the reasoning process traceable and verifiable. To enable this capability, we first construct high-quality datasets STGR that provide unified spatio-temporal supervision, which is absent in existing resources. We further adopt a cold-start reinforcement learning strategy with specially designed rewards that jointly encourage answer accuracy, temporal alignment, and spatial precision. On the V-STAR benchmark, Open-o3-Video achieves state-of-the-art performance, improving mAM by 14.4% and mLGM by 24.2% over the Qwen2.5-VL baseline, and shows consistent gains across a range of video understanding benchmarks. Beyond accuracy, the grounded reasoning traces produced by Open-o3-Video support confidence-aware test-time scaling, improving answer reliability. The code, model and datasets are publicly available at https://marinero4972.github.io/projects/Open-o3-Video/.} }
Endnote
%0 Conference Paper %T Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence %A Jiahao Meng %A Xiangtai Li %A Haochen Wang %A Tan Yue %A Tao Zhang %A Lingdong Kong %A Yunhai Tong %A Anran Wang %A Zhiyang Teng %A Yujing Wang %A Zhuochen Wang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-meng26e %I PMLR %P 87958--87979 %U https://proceedings.mlr.press/v306/meng26e.html %V 306 %X Most video reasoning models only generate textual reasoning traces without indicating when and where key evidence appears. Recent models such as OpenAI-o3 have sparked wide interest in evidence-centered reasoning for images, yet extending this ability to videos is more challenging due to the need for joint temporal tracking and spatial localization across dynamic scenes. We introduce Open-o3-Video, a non-agent framework that integrates explicit spatio-temporal evidence into video reasoning by highlighting key timestamps, objects, and bounding boxes, making the reasoning process traceable and verifiable. To enable this capability, we first construct high-quality datasets STGR that provide unified spatio-temporal supervision, which is absent in existing resources. We further adopt a cold-start reinforcement learning strategy with specially designed rewards that jointly encourage answer accuracy, temporal alignment, and spatial precision. On the V-STAR benchmark, Open-o3-Video achieves state-of-the-art performance, improving mAM by 14.4% and mLGM by 24.2% over the Qwen2.5-VL baseline, and shows consistent gains across a range of video understanding benchmarks. Beyond accuracy, the grounded reasoning traces produced by Open-o3-Video support confidence-aware test-time scaling, improving answer reliability. The code, model and datasets are publicly available at https://marinero4972.github.io/projects/Open-o3-Video/.
APA
Meng, J., Li, X., Wang, H., Yue, T., Zhang, T., Kong, L., Tong, Y., Wang, A., Teng, Z., Wang, Y. & Wang, Z.. (2026). Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:87958-87979 Available from https://proceedings.mlr.press/v306/meng26e.html.

Related Material