Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces

Xingchen Wan, Vu Nguyen, Huong Ha, Binxin Ru, Cong Lu, Michael A. Osborne
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:10663-10674, 2021.

Abstract

High-dimensional black-box optimisation remains an important yet notoriously challenging problem. Despite the success of Bayesian optimisation methods on continuous domains, domains that are categorical, or that mix continuous and categorical variables, remain challenging. We propose a novel solution—we combine local optimisation with a tailored kernel design, effectively handling high-dimensional categorical and mixed search spaces, whilst retaining sample efficiency. We further derive convergence guarantee for the proposed approach. Finally, we demonstrate empirically that our method outperforms the current baselines on a variety of synthetic and real-world tasks in terms of performance, computational costs, or both.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-wan21b, title = {Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces}, author = {Wan, Xingchen and Nguyen, Vu and Ha, Huong and Ru, Binxin and Lu, Cong and Osborne, Michael A.}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {10663--10674}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/wan21b/wan21b.pdf}, url = {https://proceedings.mlr.press/v139/wan21b.html}, abstract = {High-dimensional black-box optimisation remains an important yet notoriously challenging problem. Despite the success of Bayesian optimisation methods on continuous domains, domains that are categorical, or that mix continuous and categorical variables, remain challenging. We propose a novel solution—we combine local optimisation with a tailored kernel design, effectively handling high-dimensional categorical and mixed search spaces, whilst retaining sample efficiency. We further derive convergence guarantee for the proposed approach. Finally, we demonstrate empirically that our method outperforms the current baselines on a variety of synthetic and real-world tasks in terms of performance, computational costs, or both.} }
Endnote
%0 Conference Paper %T Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces %A Xingchen Wan %A Vu Nguyen %A Huong Ha %A Binxin Ru %A Cong Lu %A Michael A. Osborne %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-wan21b %I PMLR %P 10663--10674 %U https://proceedings.mlr.press/v139/wan21b.html %V 139 %X High-dimensional black-box optimisation remains an important yet notoriously challenging problem. Despite the success of Bayesian optimisation methods on continuous domains, domains that are categorical, or that mix continuous and categorical variables, remain challenging. We propose a novel solution—we combine local optimisation with a tailored kernel design, effectively handling high-dimensional categorical and mixed search spaces, whilst retaining sample efficiency. We further derive convergence guarantee for the proposed approach. Finally, we demonstrate empirically that our method outperforms the current baselines on a variety of synthetic and real-world tasks in terms of performance, computational costs, or both.
APA
Wan, X., Nguyen, V., Ha, H., Ru, B., Lu, C. & Osborne, M.A.. (2021). Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:10663-10674 Available from https://proceedings.mlr.press/v139/wan21b.html.

Related Material