OvisOCR: End-to-End Document Parsing via Aligning Specialized Perception with General Reasoning

Jun-Peng Jiang, Shiyin Lu, An-Yang Ji, Yinglun Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, Han-Jia Ye
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:52803-52817, 2026.

Abstract

This paper presents OvisOCR, a lightweight and strictly end-to-end Multimodal Large Language Model (MLLM) tailored for document parsing. Unlike current methods that rely on complex "Crop-OCR-Merge’" cascades to handle high-resolution inputs, OvisOCR directly maps full-page visual signals to structured Markdown without localized slicing or layout detection dependencies. Through extensive evaluations on the OmniDocBench, OvisOCR achieves top-tier performance among all compared methods in all aspects, demonstrating that a compact E2E model can effectively “digest” the capabilities of intricate pipelines and surpass specialized and general methods. Technically, OvisOCR unifies fine-grained recognition and semantic correction in a single model by leveraging supervision bootstrapped from strong OCR engines and refined via general-model-based data cleaning. To balance the performance across diverse document constituents, we design category-specific reward mechanisms for distinct element types, such as dense text, complex tables, and formulas, and ensure the model enhances its formatting strengths for each modality concurrently. This approach effectively resolves the optimization conflict, guaranteeing that improvements in structural layout parsing do not come at the expense of omitting fine-grained textual details. Empirical results confirm that OvisOCR eliminates the error propagation inherent in split-and-merge architectures, offering a streamlined path for future document intelligence. Our model is available at https://huggingface.co/ATH-MaaS/OvisOCR.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-jiang26ag, title = {{O}vis{OCR}: End-to-End Document Parsing via Aligning Specialized Perception with General Reasoning}, author = {Jiang, Jun-Peng and Lu, Shiyin and Ji, An-Yang and Li, Yinglun and Chen, Qing-Guo and Xu, Zhao and Luo, Weihua and Zhang, Kaifu and Zhan, De-Chuan and Ye, Han-Jia}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {52803--52817}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/jiang26ag/jiang26ag.pdf}, url = {https://proceedings.mlr.press/v306/jiang26ag.html}, abstract = {This paper presents OvisOCR, a lightweight and strictly end-to-end Multimodal Large Language Model (MLLM) tailored for document parsing. Unlike current methods that rely on complex "Crop-OCR-Merge’" cascades to handle high-resolution inputs, OvisOCR directly maps full-page visual signals to structured Markdown without localized slicing or layout detection dependencies. Through extensive evaluations on the OmniDocBench, OvisOCR achieves top-tier performance among all compared methods in all aspects, demonstrating that a compact E2E model can effectively “digest” the capabilities of intricate pipelines and surpass specialized and general methods. Technically, OvisOCR unifies fine-grained recognition and semantic correction in a single model by leveraging supervision bootstrapped from strong OCR engines and refined via general-model-based data cleaning. To balance the performance across diverse document constituents, we design category-specific reward mechanisms for distinct element types, such as dense text, complex tables, and formulas, and ensure the model enhances its formatting strengths for each modality concurrently. This approach effectively resolves the optimization conflict, guaranteeing that improvements in structural layout parsing do not come at the expense of omitting fine-grained textual details. Empirical results confirm that OvisOCR eliminates the error propagation inherent in split-and-merge architectures, offering a streamlined path for future document intelligence. Our model is available at https://huggingface.co/ATH-MaaS/OvisOCR.} }
Endnote
%0 Conference Paper %T OvisOCR: End-to-End Document Parsing via Aligning Specialized Perception with General Reasoning %A Jun-Peng Jiang %A Shiyin Lu %A An-Yang Ji %A Yinglun Li %A Qing-Guo Chen %A Zhao Xu %A Weihua Luo %A Kaifu Zhang %A De-Chuan Zhan %A Han-Jia Ye %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-jiang26ag %I PMLR %P 52803--52817 %U https://proceedings.mlr.press/v306/jiang26ag.html %V 306 %X This paper presents OvisOCR, a lightweight and strictly end-to-end Multimodal Large Language Model (MLLM) tailored for document parsing. Unlike current methods that rely on complex "Crop-OCR-Merge’" cascades to handle high-resolution inputs, OvisOCR directly maps full-page visual signals to structured Markdown without localized slicing or layout detection dependencies. Through extensive evaluations on the OmniDocBench, OvisOCR achieves top-tier performance among all compared methods in all aspects, demonstrating that a compact E2E model can effectively “digest” the capabilities of intricate pipelines and surpass specialized and general methods. Technically, OvisOCR unifies fine-grained recognition and semantic correction in a single model by leveraging supervision bootstrapped from strong OCR engines and refined via general-model-based data cleaning. To balance the performance across diverse document constituents, we design category-specific reward mechanisms for distinct element types, such as dense text, complex tables, and formulas, and ensure the model enhances its formatting strengths for each modality concurrently. This approach effectively resolves the optimization conflict, guaranteeing that improvements in structural layout parsing do not come at the expense of omitting fine-grained textual details. Empirical results confirm that OvisOCR eliminates the error propagation inherent in split-and-merge architectures, offering a streamlined path for future document intelligence. Our model is available at https://huggingface.co/ATH-MaaS/OvisOCR.
APA
Jiang, J., Lu, S., Ji, A., Li, Y., Chen, Q., Xu, Z., Luo, W., Zhang, K., Zhan, D. & Ye, H.. (2026). OvisOCR: End-to-End Document Parsing via Aligning Specialized Perception with General Reasoning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:52803-52817 Available from https://proceedings.mlr.press/v306/jiang26ag.html.

Related Material