Bayesian Optimization with Inequality Constraints

Jacob Gardner, Matt Kusner,  Zhixiang, Kilian Weinberger, John Cunningham
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):937-945, 2014.

Abstract

Bayesian optimization is a powerful framework for minimizing expensive objective functions while using very few function evaluations. It has been successfully applied to a variety of problems, including hyperparameter tuning and experimental design. However, this framework has not been extended to the inequality-constrained optimization setting, particularly the setting in which evaluating feasibility is just as expensive as evaluating the objective. Here we present constrained Bayesian optimization, which places a prior distribution on both the objective and the constraint functions. We evaluate our method on simulated and real data, demonstrating that constrained Bayesian optimization can quickly find optimal and feasible points, even when small feasible regions cause standard methods to fail.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-gardner14, title = {Bayesian Optimization with Inequality Constraints}, author = {Gardner, Jacob and Kusner, Matt and Zhixiang, and Weinberger, Kilian and Cunningham, John}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {937--945}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/gardner14.pdf}, url = {https://proceedings.mlr.press/v32/gardner14.html}, abstract = {Bayesian optimization is a powerful framework for minimizing expensive objective functions while using very few function evaluations. It has been successfully applied to a variety of problems, including hyperparameter tuning and experimental design. However, this framework has not been extended to the inequality-constrained optimization setting, particularly the setting in which evaluating feasibility is just as expensive as evaluating the objective. Here we present constrained Bayesian optimization, which places a prior distribution on both the objective and the constraint functions. We evaluate our method on simulated and real data, demonstrating that constrained Bayesian optimization can quickly find optimal and feasible points, even when small feasible regions cause standard methods to fail.} }
Endnote
%0 Conference Paper %T Bayesian Optimization with Inequality Constraints %A Jacob Gardner %A Matt Kusner %A Zhixiang %A Kilian Weinberger %A John Cunningham %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-gardner14 %I PMLR %P 937--945 %U https://proceedings.mlr.press/v32/gardner14.html %V 32 %N 2 %X Bayesian optimization is a powerful framework for minimizing expensive objective functions while using very few function evaluations. It has been successfully applied to a variety of problems, including hyperparameter tuning and experimental design. However, this framework has not been extended to the inequality-constrained optimization setting, particularly the setting in which evaluating feasibility is just as expensive as evaluating the objective. Here we present constrained Bayesian optimization, which places a prior distribution on both the objective and the constraint functions. We evaluate our method on simulated and real data, demonstrating that constrained Bayesian optimization can quickly find optimal and feasible points, even when small feasible regions cause standard methods to fail.
RIS
TY - CPAPER TI - Bayesian Optimization with Inequality Constraints AU - Jacob Gardner AU - Matt Kusner AU - Zhixiang AU - Kilian Weinberger AU - John Cunningham BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-gardner14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 937 EP - 945 L1 - http://proceedings.mlr.press/v32/gardner14.pdf UR - https://proceedings.mlr.press/v32/gardner14.html AB - Bayesian optimization is a powerful framework for minimizing expensive objective functions while using very few function evaluations. It has been successfully applied to a variety of problems, including hyperparameter tuning and experimental design. However, this framework has not been extended to the inequality-constrained optimization setting, particularly the setting in which evaluating feasibility is just as expensive as evaluating the objective. Here we present constrained Bayesian optimization, which places a prior distribution on both the objective and the constraint functions. We evaluate our method on simulated and real data, demonstrating that constrained Bayesian optimization can quickly find optimal and feasible points, even when small feasible regions cause standard methods to fail. ER -
APA
Gardner, J., Kusner, M., Zhixiang, , Weinberger, K. & Cunningham, J.. (2014). Bayesian Optimization with Inequality Constraints. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):937-945 Available from https://proceedings.mlr.press/v32/gardner14.html.

Related Material