BioProBench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Science

Yuyang Liu, Liuzhenghao Lv, Xiancheng Zhang, Jingya Wang, Li Yuan, Yonghong Tian
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:77772-77806, 2026.

Abstract

The realization of autonomous scientific experimentation is currently limited by LLMs’ struggle to grasp the strict procedural logic and accuracy required by biological protocols. To address this fundamental challenge, we present BioProBench, a comprehensive resource for procedural reasoning in biology. BioProBench is grounded in BioProCorpus, a foundational collection of 27,000 human-written protocols. From this corpus, we systematically constructed a dataset of over 550,000 task instances, offering both a large-scale training resource and a rigorous benchmark with novel metrics. Evaluating 10 mainstream LLMs, we find that while general comprehension is high, performance drops significantly on tasks demanding deep reasoning, quantitative precision, and safety awareness. To demonstrate the value of BioProCorpus in mitigating these issues, we developed ProAgent, grounded in our corpus, ProAgent substantially advances the state-of-the-art. Code and data are available at: https://github.com/YuyangSunshine/bioprotocolbench and https://huggingface.co/datasets/BioProBench/BioProBench .

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-liu26cu, title = {{B}io{P}ro{B}ench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Science}, author = {Liu, Yuyang and Lv, Liuzhenghao and Zhang, Xiancheng and Wang, Jingya and Yuan, Li and Tian, Yonghong}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {77772--77806}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/liu26cu/liu26cu.pdf}, url = {https://proceedings.mlr.press/v306/liu26cu.html}, abstract = {The realization of autonomous scientific experimentation is currently limited by LLMs’ struggle to grasp the strict procedural logic and accuracy required by biological protocols. To address this fundamental challenge, we present BioProBench, a comprehensive resource for procedural reasoning in biology. BioProBench is grounded in BioProCorpus, a foundational collection of 27,000 human-written protocols. From this corpus, we systematically constructed a dataset of over 550,000 task instances, offering both a large-scale training resource and a rigorous benchmark with novel metrics. Evaluating 10 mainstream LLMs, we find that while general comprehension is high, performance drops significantly on tasks demanding deep reasoning, quantitative precision, and safety awareness. To demonstrate the value of BioProCorpus in mitigating these issues, we developed ProAgent, grounded in our corpus, ProAgent substantially advances the state-of-the-art. Code and data are available at: https://github.com/YuyangSunshine/bioprotocolbench and https://huggingface.co/datasets/BioProBench/BioProBench .} }
Endnote
%0 Conference Paper %T BioProBench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Science %A Yuyang Liu %A Liuzhenghao Lv %A Xiancheng Zhang %A Jingya Wang %A Li Yuan %A Yonghong Tian %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-liu26cu %I PMLR %P 77772--77806 %U https://proceedings.mlr.press/v306/liu26cu.html %V 306 %X The realization of autonomous scientific experimentation is currently limited by LLMs’ struggle to grasp the strict procedural logic and accuracy required by biological protocols. To address this fundamental challenge, we present BioProBench, a comprehensive resource for procedural reasoning in biology. BioProBench is grounded in BioProCorpus, a foundational collection of 27,000 human-written protocols. From this corpus, we systematically constructed a dataset of over 550,000 task instances, offering both a large-scale training resource and a rigorous benchmark with novel metrics. Evaluating 10 mainstream LLMs, we find that while general comprehension is high, performance drops significantly on tasks demanding deep reasoning, quantitative precision, and safety awareness. To demonstrate the value of BioProCorpus in mitigating these issues, we developed ProAgent, grounded in our corpus, ProAgent substantially advances the state-of-the-art. Code and data are available at: https://github.com/YuyangSunshine/bioprotocolbench and https://huggingface.co/datasets/BioProBench/BioProBench .
APA
Liu, Y., Lv, L., Zhang, X., Wang, J., Yuan, L. & Tian, Y.. (2026). BioProBench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Science. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:77772-77806 Available from https://proceedings.mlr.press/v306/liu26cu.html.

Related Material