Exploiting Strategy-Space Diversity for Batch Bayesian Optimization

Sunil Gupta, Alistair Shilton, Santu Rana, Svetha Venkatesh
Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, PMLR 84:538-547, 2018.

Abstract

This paper proposes a novel approach to batch Bayesian optimisation using a multi-objective optimisation framework with exploitation and exploration forming two objectives. The key advantage of this approach is that it uses a suite of strategies to balance exploration and exploitation and thus can efficiently handle the optimisation of a variety of functions with small to large number of local extrema. Another advantage is that it automatically determines the batch size within a specified budget avoiding unnecessary function evaluations. Theoretical analysis shows that the regret not only reduces sub-linearly but also by an additional reduction factor determined by the batch size. We demonstrate the efficiency of our algorithm by optimising a variety of benchmark functions, performing hyperparameter tuning of support vector regression and classification, and finally heat treatment process of an Al-Sc alloy. Comparisons with recent baseline algorithms confirm the usefulness of our algorithm.

Cite this Paper

BibTeX
@InProceedings{pmlr-v84-gupta18a, title = {Exploiting Strategy-Space Diversity for Batch Bayesian Optimization}, author = {Gupta, Sunil and Shilton, Alistair and Rana, Santu and Venkatesh, Svetha}, booktitle = {Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics}, pages = {538--547}, year = {2018}, editor = {Storkey, Amos and Perez-Cruz, Fernando}, volume = {84}, series = {Proceedings of Machine Learning Research}, month = {09--11 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v84/gupta18a/gupta18a.pdf}, url = {https://proceedings.mlr.press/v84/gupta18a.html}, abstract = {This paper proposes a novel approach to batch Bayesian optimisation using a multi-objective optimisation framework with exploitation and exploration forming two objectives. The key advantage of this approach is that it uses a suite of strategies to balance exploration and exploitation and thus can efficiently handle the optimisation of a variety of functions with small to large number of local extrema. Another advantage is that it automatically determines the batch size within a specified budget avoiding unnecessary function evaluations. Theoretical analysis shows that the regret not only reduces sub-linearly but also by an additional reduction factor determined by the batch size. We demonstrate the efficiency of our algorithm by optimising a variety of benchmark functions, performing hyperparameter tuning of support vector regression and classification, and finally heat treatment process of an Al-Sc alloy. Comparisons with recent baseline algorithms confirm the usefulness of our algorithm.} }
Endnote
%0 Conference Paper %T Exploiting Strategy-Space Diversity for Batch Bayesian Optimization %A Sunil Gupta %A Alistair Shilton %A Santu Rana %A Svetha Venkatesh %B Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2018 %E Amos Storkey %E Fernando Perez-Cruz %F pmlr-v84-gupta18a %I PMLR %P 538--547 %U https://proceedings.mlr.press/v84/gupta18a.html %V 84 %X This paper proposes a novel approach to batch Bayesian optimisation using a multi-objective optimisation framework with exploitation and exploration forming two objectives. The key advantage of this approach is that it uses a suite of strategies to balance exploration and exploitation and thus can efficiently handle the optimisation of a variety of functions with small to large number of local extrema. Another advantage is that it automatically determines the batch size within a specified budget avoiding unnecessary function evaluations. Theoretical analysis shows that the regret not only reduces sub-linearly but also by an additional reduction factor determined by the batch size. We demonstrate the efficiency of our algorithm by optimising a variety of benchmark functions, performing hyperparameter tuning of support vector regression and classification, and finally heat treatment process of an Al-Sc alloy. Comparisons with recent baseline algorithms confirm the usefulness of our algorithm.
APA
Gupta, S., Shilton, A., Rana, S. & Venkatesh, S.. (2018). Exploiting Strategy-Space Diversity for Batch Bayesian Optimization. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 84:538-547 Available from https://proceedings.mlr.press/v84/gupta18a.html.

Related Material