Cylindrical Thompson Sampling for High-Dimensional Bayesian Optimization

Bahador Rashidi, Kerrick Johnstonbaugh, Chao Gao
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:3502-3510, 2024.

Abstract

Many industrial and scientific applications require optimization of one or more objectives by tuning dozens or hundreds of input parameters. While Bayesian optimization has been a popular approach for the efficient optimization of blackbox functions, its performance decreases drastically as the dimensionality of the search space increases (i.e., above twenty). Recent advancements in high-dimensional Bayesian optimization (HDBO) seek to mitigate this issue through techniques such as adaptive local search with trust regions or dimensionality reduction using random embeddings. In this paper, we provide a close examination of these advancements and show that sampling strategy plays a prominent role and is key to tackling the curse-of-dimensionality. We then propose cylindrical Thompson sampling (CTS), a novel strategy that can be integrated into single- and multi-objective HDBO algorithms. We demonstrate this by integrating CTS as a modular component in state-of-the-art HDBO algorithms. We verify the effectiveness of CTS on both synthetic and real-world high-dimensional problems, and show that CTS largely enhances existing HDBO methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-rashidi24a, title = {Cylindrical {T}hompson Sampling for High-Dimensional {B}ayesian Optimization}, author = {Rashidi, Bahador and Johnstonbaugh, Kerrick and Gao, Chao}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {3502--3510}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/rashidi24a/rashidi24a.pdf}, url = {https://proceedings.mlr.press/v238/rashidi24a.html}, abstract = {Many industrial and scientific applications require optimization of one or more objectives by tuning dozens or hundreds of input parameters. While Bayesian optimization has been a popular approach for the efficient optimization of blackbox functions, its performance decreases drastically as the dimensionality of the search space increases (i.e., above twenty). Recent advancements in high-dimensional Bayesian optimization (HDBO) seek to mitigate this issue through techniques such as adaptive local search with trust regions or dimensionality reduction using random embeddings. In this paper, we provide a close examination of these advancements and show that sampling strategy plays a prominent role and is key to tackling the curse-of-dimensionality. We then propose cylindrical Thompson sampling (CTS), a novel strategy that can be integrated into single- and multi-objective HDBO algorithms. We demonstrate this by integrating CTS as a modular component in state-of-the-art HDBO algorithms. We verify the effectiveness of CTS on both synthetic and real-world high-dimensional problems, and show that CTS largely enhances existing HDBO methods.} }
Endnote
%0 Conference Paper %T Cylindrical Thompson Sampling for High-Dimensional Bayesian Optimization %A Bahador Rashidi %A Kerrick Johnstonbaugh %A Chao Gao %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-rashidi24a %I PMLR %P 3502--3510 %U https://proceedings.mlr.press/v238/rashidi24a.html %V 238 %X Many industrial and scientific applications require optimization of one or more objectives by tuning dozens or hundreds of input parameters. While Bayesian optimization has been a popular approach for the efficient optimization of blackbox functions, its performance decreases drastically as the dimensionality of the search space increases (i.e., above twenty). Recent advancements in high-dimensional Bayesian optimization (HDBO) seek to mitigate this issue through techniques such as adaptive local search with trust regions or dimensionality reduction using random embeddings. In this paper, we provide a close examination of these advancements and show that sampling strategy plays a prominent role and is key to tackling the curse-of-dimensionality. We then propose cylindrical Thompson sampling (CTS), a novel strategy that can be integrated into single- and multi-objective HDBO algorithms. We demonstrate this by integrating CTS as a modular component in state-of-the-art HDBO algorithms. We verify the effectiveness of CTS on both synthetic and real-world high-dimensional problems, and show that CTS largely enhances existing HDBO methods.
APA
Rashidi, B., Johnstonbaugh, K. & Gao, C.. (2024). Cylindrical Thompson Sampling for High-Dimensional Bayesian Optimization. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:3502-3510 Available from https://proceedings.mlr.press/v238/rashidi24a.html.

Related Material