Scaling Agentic Verifier for Competitive Coding

Zeyao Ma, Jing Zhang, Xiaokang Zhang, Jiaxi Yang, Zongmeng Zhang, Jiajun Zhang, Yuheng Jing, Lei Zhang, Hao Zheng, Wenting Zhao, Junyang Lin, Binyuan Hui
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:85299-85319, 2026.

Abstract

Large language models (LLMs) have demonstrated strong coding capabilities but still struggle to solve competitive programming problems correctly in a single attempt. Execution-based re-ranking offers a promising test-time scaling strategy, yet existing methods are constrained by either difficult test case generation or inefficient random input sampling. To address this limitation, we propose Agentic Verifier, an execution-based agent that actively reasons about program behaviors and searches for highly discriminative test inputs that expose behavioral discrepancies among candidate solutions. Through multi-turn interaction with code execution environments, the verifier iteratively refines the candidate input generator and produces targeted counterexamples rather than blindly sampling inputs. We train the verifier to acquire this discriminative input generation capability via a scalable pipeline combining large-scale data synthesis, rejection fine-tuning, and agentic reinforcement learning. Extensive experiments across five competitive programming benchmarks demonstrate consistent improvements over strong execution-based baselines, achieving up to +10-15% absolute gains in Best@$k$ accuracy. Further analysis reveals clear test-time scaling behavior and highlights the verifier’s broader potential beyond reranking.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-ma26av, title = {Scaling Agentic Verifier for Competitive Coding}, author = {Ma, Zeyao and Zhang, Jing and Zhang, Xiaokang and Yang, Jiaxi and Zhang, Zongmeng and Zhang, Jiajun and Jing, Yuheng and Zhang, Lei and Zheng, Hao and Zhao, Wenting and Lin, Junyang and Hui, Binyuan}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {85299--85319}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/ma26av/ma26av.pdf}, url = {https://proceedings.mlr.press/v306/ma26av.html}, abstract = {Large language models (LLMs) have demonstrated strong coding capabilities but still struggle to solve competitive programming problems correctly in a single attempt. Execution-based re-ranking offers a promising test-time scaling strategy, yet existing methods are constrained by either difficult test case generation or inefficient random input sampling. To address this limitation, we propose Agentic Verifier, an execution-based agent that actively reasons about program behaviors and searches for highly discriminative test inputs that expose behavioral discrepancies among candidate solutions. Through multi-turn interaction with code execution environments, the verifier iteratively refines the candidate input generator and produces targeted counterexamples rather than blindly sampling inputs. We train the verifier to acquire this discriminative input generation capability via a scalable pipeline combining large-scale data synthesis, rejection fine-tuning, and agentic reinforcement learning. Extensive experiments across five competitive programming benchmarks demonstrate consistent improvements over strong execution-based baselines, achieving up to +10-15% absolute gains in Best@$k$ accuracy. Further analysis reveals clear test-time scaling behavior and highlights the verifier’s broader potential beyond reranking.} }
Endnote
%0 Conference Paper %T Scaling Agentic Verifier for Competitive Coding %A Zeyao Ma %A Jing Zhang %A Xiaokang Zhang %A Jiaxi Yang %A Zongmeng Zhang %A Jiajun Zhang %A Yuheng Jing %A Lei Zhang %A Hao Zheng %A Wenting Zhao %A Junyang Lin %A Binyuan Hui %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-ma26av %I PMLR %P 85299--85319 %U https://proceedings.mlr.press/v306/ma26av.html %V 306 %X Large language models (LLMs) have demonstrated strong coding capabilities but still struggle to solve competitive programming problems correctly in a single attempt. Execution-based re-ranking offers a promising test-time scaling strategy, yet existing methods are constrained by either difficult test case generation or inefficient random input sampling. To address this limitation, we propose Agentic Verifier, an execution-based agent that actively reasons about program behaviors and searches for highly discriminative test inputs that expose behavioral discrepancies among candidate solutions. Through multi-turn interaction with code execution environments, the verifier iteratively refines the candidate input generator and produces targeted counterexamples rather than blindly sampling inputs. We train the verifier to acquire this discriminative input generation capability via a scalable pipeline combining large-scale data synthesis, rejection fine-tuning, and agentic reinforcement learning. Extensive experiments across five competitive programming benchmarks demonstrate consistent improvements over strong execution-based baselines, achieving up to +10-15% absolute gains in Best@$k$ accuracy. Further analysis reveals clear test-time scaling behavior and highlights the verifier’s broader potential beyond reranking.
APA
Ma, Z., Zhang, J., Zhang, X., Yang, J., Zhang, Z., Zhang, J., Jing, Y., Zhang, L., Zheng, H., Zhao, W., Lin, J. & Hui, B.. (2026). Scaling Agentic Verifier for Competitive Coding. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:85299-85319 Available from https://proceedings.mlr.press/v306/ma26av.html.

Related Material