LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation

Zhuo Chen, Xinzhe Yuan, Jianshu Zhang, Jinzong Dong, Ruichen Zhou, Yingchun Niu, Tianhang Zhou, Yu Yang Fredrik Liu, Yuqiang Li, Nanyang Ye, Qinying Gu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:17979-18005, 2026.

Abstract

The high cost and data scarcity in scientific exploration have motivated the use of large language models (LLMs) as knowledge-driven components in Bayesian optimization (BO). However, existing approaches typically embed LLMs directly into the sampling or surrogate modeling pipeline, without fully leveraging their significantly lower evaluation cost compared to real-world experiments. To address this limitation, we propose LLM-Accelerated Bayesian Optimization (LABO), a framework that combines LLM predictions with experimental observations within a single BO loop. LABO employs a gating mechanism to dynamically balance reliance on LLM predictions versus actual experiments. By leveraging inexpensive LLM evaluations to broadly explore the search space and reserving costly real experiments only for regions with high uncertainty, LABO achieves more sample-efficient optimization. We provide a theoretical analysis with a cumulative regret bound that formalizes this efficiency gain. Empirical results across diverse scientific tasks demonstrate that LABO consistently outperforms existing methods under identical experimental budgets. Our results suggest that LABO offers a practical and theoretically grounded approach for integrating LLMs into scientific discovery workflows.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-chen26fx, title = {{LABO}: {LLM}-Accelerated {B}ayesian Optimization through Broad Exploration and Selective Experimentation}, author = {Chen, Zhuo and Yuan, Xinzhe and Zhang, Jianshu and Dong, Jinzong and Zhou, Ruichen and Niu, Yingchun and Zhou, Tianhang and Liu, Yu Yang Fredrik and Li, Yuqiang and Ye, Nanyang and Gu, Qinying}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {17979--18005}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/chen26fx/chen26fx.pdf}, url = {https://proceedings.mlr.press/v306/chen26fx.html}, abstract = {The high cost and data scarcity in scientific exploration have motivated the use of large language models (LLMs) as knowledge-driven components in Bayesian optimization (BO). However, existing approaches typically embed LLMs directly into the sampling or surrogate modeling pipeline, without fully leveraging their significantly lower evaluation cost compared to real-world experiments. To address this limitation, we propose LLM-Accelerated Bayesian Optimization (LABO), a framework that combines LLM predictions with experimental observations within a single BO loop. LABO employs a gating mechanism to dynamically balance reliance on LLM predictions versus actual experiments. By leveraging inexpensive LLM evaluations to broadly explore the search space and reserving costly real experiments only for regions with high uncertainty, LABO achieves more sample-efficient optimization. We provide a theoretical analysis with a cumulative regret bound that formalizes this efficiency gain. Empirical results across diverse scientific tasks demonstrate that LABO consistently outperforms existing methods under identical experimental budgets. Our results suggest that LABO offers a practical and theoretically grounded approach for integrating LLMs into scientific discovery workflows.} }
Endnote
%0 Conference Paper %T LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation %A Zhuo Chen %A Xinzhe Yuan %A Jianshu Zhang %A Jinzong Dong %A Ruichen Zhou %A Yingchun Niu %A Tianhang Zhou %A Yu Yang Fredrik Liu %A Yuqiang Li %A Nanyang Ye %A Qinying Gu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-chen26fx %I PMLR %P 17979--18005 %U https://proceedings.mlr.press/v306/chen26fx.html %V 306 %X The high cost and data scarcity in scientific exploration have motivated the use of large language models (LLMs) as knowledge-driven components in Bayesian optimization (BO). However, existing approaches typically embed LLMs directly into the sampling or surrogate modeling pipeline, without fully leveraging their significantly lower evaluation cost compared to real-world experiments. To address this limitation, we propose LLM-Accelerated Bayesian Optimization (LABO), a framework that combines LLM predictions with experimental observations within a single BO loop. LABO employs a gating mechanism to dynamically balance reliance on LLM predictions versus actual experiments. By leveraging inexpensive LLM evaluations to broadly explore the search space and reserving costly real experiments only for regions with high uncertainty, LABO achieves more sample-efficient optimization. We provide a theoretical analysis with a cumulative regret bound that formalizes this efficiency gain. Empirical results across diverse scientific tasks demonstrate that LABO consistently outperforms existing methods under identical experimental budgets. Our results suggest that LABO offers a practical and theoretically grounded approach for integrating LLMs into scientific discovery workflows.
APA
Chen, Z., Yuan, X., Zhang, J., Dong, J., Zhou, R., Niu, Y., Zhou, T., Liu, Y.Y.F., Li, Y., Ye, N. & Gu, Q.. (2026). LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:17979-18005 Available from https://proceedings.mlr.press/v306/chen26fx.html.

Related Material