Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes

Zhiyuan Jerry Lin, Raul Astudillo, Peter Frazier, Eytan Bakshy
Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, PMLR 151:4235-4258, 2022.

Abstract

We consider Bayesian optimization of expensive-to-evaluate experiments that generate vector-valued outcomes over which a decision-maker (DM) has preferences. These preferences are encoded by a utility function that is not known in closed form but can be estimated by asking the DM to express preferences over pairs of outcome vectors. To address this problem, we develop Bayesian optimization with preference exploration, a novel framework that alternates between interactive real-time preference learning with the DM via pairwise comparisons between outcomes, and Bayesian optimization with a learned compositional model of DM utility and outcomes. Within this framework, we propose preference exploration strategies specifically designed for this task, and demonstrate their performance via extensive simulation studies.

Cite this Paper

BibTeX
@InProceedings{pmlr-v151-jerry-lin22a, title = { Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes }, author = {Jerry Lin, Zhiyuan and Astudillo, Raul and Frazier, Peter and Bakshy, Eytan}, booktitle = {Proceedings of The 25th International Conference on Artificial Intelligence and Statistics}, pages = {4235--4258}, year = {2022}, editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel}, volume = {151}, series = {Proceedings of Machine Learning Research}, month = {28--30 Mar}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v151/jerry-lin22a/jerry-lin22a.pdf}, url = {https://proceedings.mlr.press/v151/jerry-lin22a.html}, abstract = { We consider Bayesian optimization of expensive-to-evaluate experiments that generate vector-valued outcomes over which a decision-maker (DM) has preferences. These preferences are encoded by a utility function that is not known in closed form but can be estimated by asking the DM to express preferences over pairs of outcome vectors. To address this problem, we develop Bayesian optimization with preference exploration, a novel framework that alternates between interactive real-time preference learning with the DM via pairwise comparisons between outcomes, and Bayesian optimization with a learned compositional model of DM utility and outcomes. Within this framework, we propose preference exploration strategies specifically designed for this task, and demonstrate their performance via extensive simulation studies. } }
Endnote
%0 Conference Paper %T Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes %A Zhiyuan Jerry Lin %A Raul Astudillo %A Peter Frazier %A Eytan Bakshy %B Proceedings of The 25th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2022 %E Gustau Camps-Valls %E Francisco J. R. Ruiz %E Isabel Valera %F pmlr-v151-jerry-lin22a %I PMLR %P 4235--4258 %U https://proceedings.mlr.press/v151/jerry-lin22a.html %V 151 %X We consider Bayesian optimization of expensive-to-evaluate experiments that generate vector-valued outcomes over which a decision-maker (DM) has preferences. These preferences are encoded by a utility function that is not known in closed form but can be estimated by asking the DM to express preferences over pairs of outcome vectors. To address this problem, we develop Bayesian optimization with preference exploration, a novel framework that alternates between interactive real-time preference learning with the DM via pairwise comparisons between outcomes, and Bayesian optimization with a learned compositional model of DM utility and outcomes. Within this framework, we propose preference exploration strategies specifically designed for this task, and demonstrate their performance via extensive simulation studies.
APA
Jerry Lin, Z., Astudillo, R., Frazier, P. & Bakshy, E.. (2022). Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes . Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 151:4235-4258 Available from https://proceedings.mlr.press/v151/jerry-lin22a.html.

Related Material