Efficient Bayesian Optimization for Target Vector Estimation

Anders Kirk Uhrenholt, Bjøern Sand Jensen
Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, PMLR 89:2661-2670, 2019.

Abstract

We consider the problem of estimating a target vector by querying an unknown multi-output function which is stochastic and expensive to evaluate. Through sequential experimental design the aim is to minimize the squared Euclidean distance between the output of the function and the target vector. Applying standard single-objective Bayesian optimization to this problem is both wasteful, since individual output components are never observed, and imprecise since the predictive distribution for new inputs will be symmetric and have negative support. We address this issue by proposing a Gaussian process model that considers the individual function outputs and derive a distribution over the resulting 2-norm. Furthermore we derive computationally efficient acquisition functions and evaluate the resulting optimization framework on several synthetic problems and a real-world problem. The results demonstrate a significant improvement over Bayesian optimization based on both standard and warped Gaussian processes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v89-uhrenholt19a, title = {Efficient Bayesian Optimization for Target Vector Estimation}, author = {Uhrenholt, Anders Kirk and Jensen, Bj{\o}ern Sand}, booktitle = {Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics}, pages = {2661--2670}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/uhrenholt19a/uhrenholt19a.pdf}, url = {https://proceedings.mlr.press/v89/uhrenholt19a.html}, abstract = {We consider the problem of estimating a target vector by querying an unknown multi-output function which is stochastic and expensive to evaluate. Through sequential experimental design the aim is to minimize the squared Euclidean distance between the output of the function and the target vector. Applying standard single-objective Bayesian optimization to this problem is both wasteful, since individual output components are never observed, and imprecise since the predictive distribution for new inputs will be symmetric and have negative support. We address this issue by proposing a Gaussian process model that considers the individual function outputs and derive a distribution over the resulting 2-norm. Furthermore we derive computationally efficient acquisition functions and evaluate the resulting optimization framework on several synthetic problems and a real-world problem. The results demonstrate a significant improvement over Bayesian optimization based on both standard and warped Gaussian processes.} }
Endnote
%0 Conference Paper %T Efficient Bayesian Optimization for Target Vector Estimation %A Anders Kirk Uhrenholt %A Bjøern Sand Jensen %B Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Masashi Sugiyama %F pmlr-v89-uhrenholt19a %I PMLR %P 2661--2670 %U https://proceedings.mlr.press/v89/uhrenholt19a.html %V 89 %X We consider the problem of estimating a target vector by querying an unknown multi-output function which is stochastic and expensive to evaluate. Through sequential experimental design the aim is to minimize the squared Euclidean distance between the output of the function and the target vector. Applying standard single-objective Bayesian optimization to this problem is both wasteful, since individual output components are never observed, and imprecise since the predictive distribution for new inputs will be symmetric and have negative support. We address this issue by proposing a Gaussian process model that considers the individual function outputs and derive a distribution over the resulting 2-norm. Furthermore we derive computationally efficient acquisition functions and evaluate the resulting optimization framework on several synthetic problems and a real-world problem. The results demonstrate a significant improvement over Bayesian optimization based on both standard and warped Gaussian processes.
APA
Uhrenholt, A.K. & Jensen, B.S.. (2019). Efficient Bayesian Optimization for Target Vector Estimation. Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 89:2661-2670 Available from https://proceedings.mlr.press/v89/uhrenholt19a.html.

Related Material