FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearch

Virginia Aglietti, Ira Ktena, Jessica Schrouff, Eleni Sgouritsa, Francisco Ruiz, Alan Malek, Alexis Bellot, Silvia Chiappa
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:639-661, 2025.

Abstract

The sample efficiency of Bayesian optimization algorithms depends on carefully crafted acquisition functions (AFs) guiding the sequential collection of function evaluations. The best-performing AFs can vary significantly across optimization problems, often requiring ad-hoc and problem-specific choices. This work tackles the challenge of designing novel AFs that perform well across a variety of experimental settings. Based on FunSearch, a recent work using Large Language Models (LLMs) for discovery in mathematical sciences, we propose FunBO, an LLM-based method that can be used to learn new AFs written in computer code by leveraging access to a number of evaluations for a limited set of objective functions. We provide the analytic expression of all discovered AFs and evaluate them on various global optimization benchmarks and hyperparameter optimization tasks. We show how FunBO identifies AFs that generalize well both in and out of the training distribution of functions, thus outperforming established general-purpose AFs and achieving competitive performance against AFs that are customized to specific function types and are learned via transfer-learning algorithms.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-aglietti25a, title = {{F}un{BO}: Discovering Acquisition Functions for {B}ayesian Optimization with {F}un{S}earch}, author = {Aglietti, Virginia and Ktena, Ira and Schrouff, Jessica and Sgouritsa, Eleni and Ruiz, Francisco and Malek, Alan and Bellot, Alexis and Chiappa, Silvia}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {639--661}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/aglietti25a/aglietti25a.pdf}, url = {https://proceedings.mlr.press/v267/aglietti25a.html}, abstract = {The sample efficiency of Bayesian optimization algorithms depends on carefully crafted acquisition functions (AFs) guiding the sequential collection of function evaluations. The best-performing AFs can vary significantly across optimization problems, often requiring ad-hoc and problem-specific choices. This work tackles the challenge of designing novel AFs that perform well across a variety of experimental settings. Based on FunSearch, a recent work using Large Language Models (LLMs) for discovery in mathematical sciences, we propose FunBO, an LLM-based method that can be used to learn new AFs written in computer code by leveraging access to a number of evaluations for a limited set of objective functions. We provide the analytic expression of all discovered AFs and evaluate them on various global optimization benchmarks and hyperparameter optimization tasks. We show how FunBO identifies AFs that generalize well both in and out of the training distribution of functions, thus outperforming established general-purpose AFs and achieving competitive performance against AFs that are customized to specific function types and are learned via transfer-learning algorithms.} }
Endnote
%0 Conference Paper %T FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearch %A Virginia Aglietti %A Ira Ktena %A Jessica Schrouff %A Eleni Sgouritsa %A Francisco Ruiz %A Alan Malek %A Alexis Bellot %A Silvia Chiappa %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-aglietti25a %I PMLR %P 639--661 %U https://proceedings.mlr.press/v267/aglietti25a.html %V 267 %X The sample efficiency of Bayesian optimization algorithms depends on carefully crafted acquisition functions (AFs) guiding the sequential collection of function evaluations. The best-performing AFs can vary significantly across optimization problems, often requiring ad-hoc and problem-specific choices. This work tackles the challenge of designing novel AFs that perform well across a variety of experimental settings. Based on FunSearch, a recent work using Large Language Models (LLMs) for discovery in mathematical sciences, we propose FunBO, an LLM-based method that can be used to learn new AFs written in computer code by leveraging access to a number of evaluations for a limited set of objective functions. We provide the analytic expression of all discovered AFs and evaluate them on various global optimization benchmarks and hyperparameter optimization tasks. We show how FunBO identifies AFs that generalize well both in and out of the training distribution of functions, thus outperforming established general-purpose AFs and achieving competitive performance against AFs that are customized to specific function types and are learned via transfer-learning algorithms.
APA
Aglietti, V., Ktena, I., Schrouff, J., Sgouritsa, E., Ruiz, F., Malek, A., Bellot, A. & Chiappa, S.. (2025). FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearch. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:639-661 Available from https://proceedings.mlr.press/v267/aglietti25a.html.

Related Material