Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selection

Yihang Shen, Carl Kingsford
Proceedings of the Second International Conference on Automated Machine Learning, PMLR 224:15/1-27, 2023.

Abstract

Bayesian Optimization (BO) is a widely-used method for the global optimization of black-box functions. While BO has been successfully applied to many scenarios, scaling BO algorithms to high-dimensional domains remains a challenge. Optimizing such functions by vanilla BO is extremely time-consuming. Alternative strategies for high-dimensional BO that are based on the idea of embedding the high-dimensional space to one with low dimensions are sensitive to the choice of the embedding dimension, which needs to be pre-specified. We develop a new computationally efficient high-dimensional BO method that leverages variable selection. We analyze the computational complexity of our algorithm and demonstrate its efficacy on several synthetic and real problems through empirical evaluations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v224-shen23a, title = {Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selection}, author = {Shen, Yihang and Kingsford, Carl}, booktitle = {Proceedings of the Second International Conference on Automated Machine Learning}, pages = {15/1--27}, year = {2023}, editor = {Faust, Aleksandra and Garnett, Roman and White, Colin and Hutter, Frank and Gardner, Jacob R.}, volume = {224}, series = {Proceedings of Machine Learning Research}, month = {12--15 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v224/shen23a/shen23a.pdf}, url = {https://proceedings.mlr.press/v224/shen23a.html}, abstract = {Bayesian Optimization (BO) is a widely-used method for the global optimization of black-box functions. While BO has been successfully applied to many scenarios, scaling BO algorithms to high-dimensional domains remains a challenge. Optimizing such functions by vanilla BO is extremely time-consuming. Alternative strategies for high-dimensional BO that are based on the idea of embedding the high-dimensional space to one with low dimensions are sensitive to the choice of the embedding dimension, which needs to be pre-specified. We develop a new computationally efficient high-dimensional BO method that leverages variable selection. We analyze the computational complexity of our algorithm and demonstrate its efficacy on several synthetic and real problems through empirical evaluations.} }
Endnote
%0 Conference Paper %T Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selection %A Yihang Shen %A Carl Kingsford %B Proceedings of the Second International Conference on Automated Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Aleksandra Faust %E Roman Garnett %E Colin White %E Frank Hutter %E Jacob R. Gardner %F pmlr-v224-shen23a %I PMLR %P 15/1--27 %U https://proceedings.mlr.press/v224/shen23a.html %V 224 %X Bayesian Optimization (BO) is a widely-used method for the global optimization of black-box functions. While BO has been successfully applied to many scenarios, scaling BO algorithms to high-dimensional domains remains a challenge. Optimizing such functions by vanilla BO is extremely time-consuming. Alternative strategies for high-dimensional BO that are based on the idea of embedding the high-dimensional space to one with low dimensions are sensitive to the choice of the embedding dimension, which needs to be pre-specified. We develop a new computationally efficient high-dimensional BO method that leverages variable selection. We analyze the computational complexity of our algorithm and demonstrate its efficacy on several synthetic and real problems through empirical evaluations.
APA
Shen, Y. & Kingsford, C.. (2023). Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selection. Proceedings of the Second International Conference on Automated Machine Learning, in Proceedings of Machine Learning Research 224:15/1-27 Available from https://proceedings.mlr.press/v224/shen23a.html.

Related Material