Bayesian Optimization with Inequality Constraints

Jacob Gardner, Matt Kusner,  Zhixiang, Kilian Weinberger, John Cunningham
; Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):937-945, 2014.

Abstract

Bayesian optimization is a powerful framework for minimizing expensive objective functions while using very few function evaluations. It has been successfully applied to a variety of problems, including hyperparameter tuning and experimental design. However, this framework has not been extended to the inequality-constrained optimization setting, particularly the setting in which evaluating feasibility is just as expensive as evaluating the objective. Here we present constrained Bayesian optimization, which places a prior distribution on both the objective and the constraint functions. We evaluate our method on simulated and real data, demonstrating that constrained Bayesian optimization can quickly find optimal and feasible points, even when small feasible regions cause standard methods to fail.

Cite this Paper


BibTeX
@InProceedings{pmlr-v32-gardner14, title = {Bayesian Optimization with Inequality Constraints}, author = {Jacob Gardner and Matt Kusner and Zhixiang and Kilian Weinberger and John Cunningham}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {937--945}, year = {2014}, editor = {Eric P. Xing and Tony Jebara}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/gardner14.pdf}, url = {http://proceedings.mlr.press/v32/gardner14.html}, abstract = {Bayesian optimization is a powerful framework for minimizing expensive objective functions while using very few function evaluations. It has been successfully applied to a variety of problems, including hyperparameter tuning and experimental design. However, this framework has not been extended to the inequality-constrained optimization setting, particularly the setting in which evaluating feasibility is just as expensive as evaluating the objective. Here we present constrained Bayesian optimization, which places a prior distribution on both the objective and the constraint functions. We evaluate our method on simulated and real data, demonstrating that constrained Bayesian optimization can quickly find optimal and feasible points, even when small feasible regions cause standard methods to fail.} }
Endnote
%0 Conference Paper %T Bayesian Optimization with Inequality Constraints %A Jacob Gardner %A Matt Kusner %A Zhixiang %A Kilian Weinberger %A John Cunningham %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-gardner14 %I PMLR %J Proceedings of Machine Learning Research %P 937--945 %U http://proceedings.mlr.press %V 32 %N 2 %W PMLR %X Bayesian optimization is a powerful framework for minimizing expensive objective functions while using very few function evaluations. It has been successfully applied to a variety of problems, including hyperparameter tuning and experimental design. However, this framework has not been extended to the inequality-constrained optimization setting, particularly the setting in which evaluating feasibility is just as expensive as evaluating the objective. Here we present constrained Bayesian optimization, which places a prior distribution on both the objective and the constraint functions. We evaluate our method on simulated and real data, demonstrating that constrained Bayesian optimization can quickly find optimal and feasible points, even when small feasible regions cause standard methods to fail.
RIS
TY - CPAPER TI - Bayesian Optimization with Inequality Constraints AU - Jacob Gardner AU - Matt Kusner AU - Zhixiang AU - Kilian Weinberger AU - John Cunningham BT - Proceedings of the 31st International Conference on Machine Learning PY - 2014/01/27 DA - 2014/01/27 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-gardner14 PB - PMLR SP - 937 DP - PMLR EP - 945 L1 - http://proceedings.mlr.press/v32/gardner14.pdf UR - http://proceedings.mlr.press/v32/gardner14.html AB - Bayesian optimization is a powerful framework for minimizing expensive objective functions while using very few function evaluations. It has been successfully applied to a variety of problems, including hyperparameter tuning and experimental design. However, this framework has not been extended to the inequality-constrained optimization setting, particularly the setting in which evaluating feasibility is just as expensive as evaluating the objective. Here we present constrained Bayesian optimization, which places a prior distribution on both the objective and the constraint functions. We evaluate our method on simulated and real data, demonstrating that constrained Bayesian optimization can quickly find optimal and feasible points, even when small feasible regions cause standard methods to fail. ER -
APA
Gardner, J., Kusner, M., Zhixiang, , Weinberger, K. & Cunningham, J.. (2014). Bayesian Optimization with Inequality Constraints. Proceedings of the 31st International Conference on Machine Learning, in PMLR 32(2):937-945

Related Material