Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimation

Fengxue Zhang, Jialin Song, James C Bowden, Alexander Ladd, Yisong Yue, Thomas Desautels, Yuxin Chen
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:41579-41595, 2023.

Abstract

We study Bayesian optimization (BO) in high-dimensional and non-stationary scenarios. Existing algorithms for such scenarios typically require extensive hyperparameter tuning, which limits their practical effectiveness. We propose a framework, called BALLET, which adaptively filters for a high-confidence region of interest (ROI) as a superlevel-set of a nonparametric probabilistic model such as a Gaussian process (GP). Our approach is easy to tune, and is able to focus on local region of the optimization space that can be tackled by existing BO methods. The key idea is to use two probabilistic models: a coarse GP to identify the ROI, and a localized GP for optimization within the ROI. We show theoretically that BALLET can efficiently shrink the search space, and can exhibit a tighter regret bound than standard BO without ROI filtering. We demonstrate empirically the effectiveness of BALLET on both synthetic and real-world optimization tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-zhang23aj, title = {Learning Regions of Interest for {B}ayesian Optimization with Adaptive Level-Set Estimation}, author = {Zhang, Fengxue and Song, Jialin and Bowden, James C and Ladd, Alexander and Yue, Yisong and Desautels, Thomas and Chen, Yuxin}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {41579--41595}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/zhang23aj/zhang23aj.pdf}, url = {https://proceedings.mlr.press/v202/zhang23aj.html}, abstract = {We study Bayesian optimization (BO) in high-dimensional and non-stationary scenarios. Existing algorithms for such scenarios typically require extensive hyperparameter tuning, which limits their practical effectiveness. We propose a framework, called BALLET, which adaptively filters for a high-confidence region of interest (ROI) as a superlevel-set of a nonparametric probabilistic model such as a Gaussian process (GP). Our approach is easy to tune, and is able to focus on local region of the optimization space that can be tackled by existing BO methods. The key idea is to use two probabilistic models: a coarse GP to identify the ROI, and a localized GP for optimization within the ROI. We show theoretically that BALLET can efficiently shrink the search space, and can exhibit a tighter regret bound than standard BO without ROI filtering. We demonstrate empirically the effectiveness of BALLET on both synthetic and real-world optimization tasks.} }
Endnote
%0 Conference Paper %T Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimation %A Fengxue Zhang %A Jialin Song %A James C Bowden %A Alexander Ladd %A Yisong Yue %A Thomas Desautels %A Yuxin Chen %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-zhang23aj %I PMLR %P 41579--41595 %U https://proceedings.mlr.press/v202/zhang23aj.html %V 202 %X We study Bayesian optimization (BO) in high-dimensional and non-stationary scenarios. Existing algorithms for such scenarios typically require extensive hyperparameter tuning, which limits their practical effectiveness. We propose a framework, called BALLET, which adaptively filters for a high-confidence region of interest (ROI) as a superlevel-set of a nonparametric probabilistic model such as a Gaussian process (GP). Our approach is easy to tune, and is able to focus on local region of the optimization space that can be tackled by existing BO methods. The key idea is to use two probabilistic models: a coarse GP to identify the ROI, and a localized GP for optimization within the ROI. We show theoretically that BALLET can efficiently shrink the search space, and can exhibit a tighter regret bound than standard BO without ROI filtering. We demonstrate empirically the effectiveness of BALLET on both synthetic and real-world optimization tasks.
APA
Zhang, F., Song, J., Bowden, J.C., Ladd, A., Yue, Y., Desautels, T. & Chen, Y.. (2023). Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimation. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:41579-41595 Available from https://proceedings.mlr.press/v202/zhang23aj.html.

Related Material