Multi-objective Bayesian Optimization using Pareto-frontier Entropy

Shinya Suzuki, Shion Takeno, Tomoyuki Tamura, Kazuki Shitara, Masayuki Karasuyama
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:9279-9288, 2020.

Abstract

This paper studies an entropy-based multi-objective Bayesian optimization (MBO). Existing entropy-based MBO methods need complicated approximations to evaluate entropy or employ over-simplification that ignores trade-off among objectives. We propose a novel entropy-based MBO called Pareto-frontier entropy search (PFES), which is based on the information gain of Pareto-frontier. We show that our entropy evaluation can be reduced to a closed form whose computation is quite simple while capturing the trade-off relation in Pareto-frontier. We further propose an extension for the “decoupled” setting, in which each objective function can be observed separately, and show that the PFES-based approach derives a natural extension of the original acquisition function which can also be evaluated simply. Our numerical experiments show effectiveness of PFES through several benchmark datasets, and real-word datasets from materials science.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-suzuki20a, title = {Multi-objective {B}ayesian Optimization using Pareto-frontier Entropy}, author = {Suzuki, Shinya and Takeno, Shion and Tamura, Tomoyuki and Shitara, Kazuki and Karasuyama, Masayuki}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {9279--9288}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/suzuki20a/suzuki20a.pdf}, url = {https://proceedings.mlr.press/v119/suzuki20a.html}, abstract = {This paper studies an entropy-based multi-objective Bayesian optimization (MBO). Existing entropy-based MBO methods need complicated approximations to evaluate entropy or employ over-simplification that ignores trade-off among objectives. We propose a novel entropy-based MBO called Pareto-frontier entropy search (PFES), which is based on the information gain of Pareto-frontier. We show that our entropy evaluation can be reduced to a closed form whose computation is quite simple while capturing the trade-off relation in Pareto-frontier. We further propose an extension for the “decoupled” setting, in which each objective function can be observed separately, and show that the PFES-based approach derives a natural extension of the original acquisition function which can also be evaluated simply. Our numerical experiments show effectiveness of PFES through several benchmark datasets, and real-word datasets from materials science.} }
Endnote
%0 Conference Paper %T Multi-objective Bayesian Optimization using Pareto-frontier Entropy %A Shinya Suzuki %A Shion Takeno %A Tomoyuki Tamura %A Kazuki Shitara %A Masayuki Karasuyama %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-suzuki20a %I PMLR %P 9279--9288 %U https://proceedings.mlr.press/v119/suzuki20a.html %V 119 %X This paper studies an entropy-based multi-objective Bayesian optimization (MBO). Existing entropy-based MBO methods need complicated approximations to evaluate entropy or employ over-simplification that ignores trade-off among objectives. We propose a novel entropy-based MBO called Pareto-frontier entropy search (PFES), which is based on the information gain of Pareto-frontier. We show that our entropy evaluation can be reduced to a closed form whose computation is quite simple while capturing the trade-off relation in Pareto-frontier. We further propose an extension for the “decoupled” setting, in which each objective function can be observed separately, and show that the PFES-based approach derives a natural extension of the original acquisition function which can also be evaluated simply. Our numerical experiments show effectiveness of PFES through several benchmark datasets, and real-word datasets from materials science.
APA
Suzuki, S., Takeno, S., Tamura, T., Shitara, K. & Karasuyama, M.. (2020). Multi-objective Bayesian Optimization using Pareto-frontier Entropy. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:9279-9288 Available from https://proceedings.mlr.press/v119/suzuki20a.html.

Related Material