daVinci-Dev: Agent-native Mid-training for Software Engineering

Ji Zeng, Dayuan Fu, Tiantian Mi, Zhuang Yumin, Yaxing Huang, Xuefeng Li, Lyumanshan Ye, Muhang Xie, Qishuo Hua, Zhen Huang, Mohan Jiang, Hanning Wang, Jifan Lin, Yang Xiao, Jie Sun, Yunze Wu, Pengfei Liu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:153265-153282, 2026.

Abstract

While the emerging field of agentic software engineering has spurred extensive research into post-training, this paradigm alone does not fully address the distribution mismatch between traditional static pre-training and dynamic deployment environments. In this paper, we instead investigate agentic mid-training as a scalable complementary approach. Central to our approach is agent-native data comprising two complementary components: contextually-native trajectories that preserve the complete information flow an agent experiences, offering broad coverage and diversity; and environmentally-native trajectories whose observations stem from actual tool invocations and test executions, providing interaction authenticity. On SWE-Bench Verified, our recipe outperforms the previous open software engineering mid-training recipe Kimi-Dev under two post-training settings with the same base model and agentic scaffold, while using fewer than half mid-training tokens (73.1B). Furthermore, our 32B and 72B models achieve state-of-the-art resolution rates of 56.1% and 58.5% among open agentic recipes using agentic scaffolds, despite starting from non-coder Qwen2.5 base models. We also observe performance gains on general code generation and scientific benchmarks. We open-source a significant portion of our datasets, recipes, and model checkpoints to facilitate further research.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-zeng26d, title = {da{V}inci-Dev: Agent-native Mid-training for Software Engineering}, author = {Zeng, Ji and Fu, Dayuan and Mi, Tiantian and Yumin, Zhuang and Huang, Yaxing and Li, Xuefeng and Ye, Lyumanshan and Xie, Muhang and Hua, Qishuo and Huang, Zhen and Jiang, Mohan and Wang, Hanning and Lin, Jifan and Xiao, Yang and Sun, Jie and Wu, Yunze and Liu, Pengfei}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {153265--153282}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/zeng26d/zeng26d.pdf}, url = {https://proceedings.mlr.press/v306/zeng26d.html}, abstract = {While the emerging field of agentic software engineering has spurred extensive research into post-training, this paradigm alone does not fully address the distribution mismatch between traditional static pre-training and dynamic deployment environments. In this paper, we instead investigate agentic mid-training as a scalable complementary approach. Central to our approach is agent-native data comprising two complementary components: contextually-native trajectories that preserve the complete information flow an agent experiences, offering broad coverage and diversity; and environmentally-native trajectories whose observations stem from actual tool invocations and test executions, providing interaction authenticity. On SWE-Bench Verified, our recipe outperforms the previous open software engineering mid-training recipe Kimi-Dev under two post-training settings with the same base model and agentic scaffold, while using fewer than half mid-training tokens (73.1B). Furthermore, our 32B and 72B models achieve state-of-the-art resolution rates of 56.1% and 58.5% among open agentic recipes using agentic scaffolds, despite starting from non-coder Qwen2.5 base models. We also observe performance gains on general code generation and scientific benchmarks. We open-source a significant portion of our datasets, recipes, and model checkpoints to facilitate further research.} }
Endnote
%0 Conference Paper %T daVinci-Dev: Agent-native Mid-training for Software Engineering %A Ji Zeng %A Dayuan Fu %A Tiantian Mi %A Zhuang Yumin %A Yaxing Huang %A Xuefeng Li %A Lyumanshan Ye %A Muhang Xie %A Qishuo Hua %A Zhen Huang %A Mohan Jiang %A Hanning Wang %A Jifan Lin %A Yang Xiao %A Jie Sun %A Yunze Wu %A Pengfei Liu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-zeng26d %I PMLR %P 153265--153282 %U https://proceedings.mlr.press/v306/zeng26d.html %V 306 %X While the emerging field of agentic software engineering has spurred extensive research into post-training, this paradigm alone does not fully address the distribution mismatch between traditional static pre-training and dynamic deployment environments. In this paper, we instead investigate agentic mid-training as a scalable complementary approach. Central to our approach is agent-native data comprising two complementary components: contextually-native trajectories that preserve the complete information flow an agent experiences, offering broad coverage and diversity; and environmentally-native trajectories whose observations stem from actual tool invocations and test executions, providing interaction authenticity. On SWE-Bench Verified, our recipe outperforms the previous open software engineering mid-training recipe Kimi-Dev under two post-training settings with the same base model and agentic scaffold, while using fewer than half mid-training tokens (73.1B). Furthermore, our 32B and 72B models achieve state-of-the-art resolution rates of 56.1% and 58.5% among open agentic recipes using agentic scaffolds, despite starting from non-coder Qwen2.5 base models. We also observe performance gains on general code generation and scientific benchmarks. We open-source a significant portion of our datasets, recipes, and model checkpoints to facilitate further research.
APA
Zeng, J., Fu, D., Mi, T., Yumin, Z., Huang, Y., Li, X., Ye, L., Xie, M., Hua, Q., Huang, Z., Jiang, M., Wang, H., Lin, J., Xiao, Y., Sun, J., Wu, Y. & Liu, P.. (2026). daVinci-Dev: Agent-native Mid-training for Software Engineering. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:153265-153282 Available from https://proceedings.mlr.press/v306/zeng26d.html.

Related Material