Bayesian Optimization with Unknown Constraints

Michael Gelbart Harvard University, Jasper Snoek Harvard University, Ryan Adams Harvard
Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, PMLR R12:46-55, 2014.

Abstract

Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general case that noise may be present in the constraint func- tions, and the objective and constraints may be evaluated independently. We provide motivating practical examples, and present a general frame- work to solve such problems. We demonstrate the effectiveness of our approach on optimizing the performance of online latent Dirichlet allo- cation subject to topic sparsity constraints, tun- ing a neural network given test-time memory constraints, and optimizing Hamiltonian Monte Carlo to achieve maximal effectiveness in a fixed time, subject to passing standard convergence di- agnostics.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR12-university14b, title = {{B}ayesian Optimization with Unknown Constraints}, author = {University, Michael Gelbart Harvard and University, Jasper Snoek Harvard and Harvard, Ryan Adams}, booktitle = {Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence}, pages = {46--55}, year = {2014}, editor = {Zhang, Nevin L. and Tian, Jin}, volume = {R12}, series = {Proceedings of Machine Learning Research}, month = {23--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r12/main/assets/university14b/university14b.pdf}, url = {https://proceedings.mlr.press/r12/university14b.html}, abstract = {Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general case that noise may be present in the constraint func- tions, and the objective and constraints may be evaluated independently. We provide motivating practical examples, and present a general frame- work to solve such problems. We demonstrate the effectiveness of our approach on optimizing the performance of online latent Dirichlet allo- cation subject to topic sparsity constraints, tun- ing a neural network given test-time memory constraints, and optimizing Hamiltonian Monte Carlo to achieve maximal effectiveness in a fixed time, subject to passing standard convergence di- agnostics.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Bayesian Optimization with Unknown Constraints %A Michael Gelbart Harvard University %A Jasper Snoek Harvard University %A Ryan Adams Harvard %B Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2014 %E Nevin L. Zhang %E Jin Tian %F pmlr-vR12-university14b %I PMLR %P 46--55 %U https://proceedings.mlr.press/r12/university14b.html %V R12 %X Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general case that noise may be present in the constraint func- tions, and the objective and constraints may be evaluated independently. We provide motivating practical examples, and present a general frame- work to solve such problems. We demonstrate the effectiveness of our approach on optimizing the performance of online latent Dirichlet allo- cation subject to topic sparsity constraints, tun- ing a neural network given test-time memory constraints, and optimizing Hamiltonian Monte Carlo to achieve maximal effectiveness in a fixed time, subject to passing standard convergence di- agnostics. %Z Reissued by PMLR on 04 October 2026.
APA
University, M.G.H., University, J.S.H. & Harvard, R.A.. (2014). Bayesian Optimization with Unknown Constraints. Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R12:46-55 Available from https://proceedings.mlr.press/r12/university14b.html. Reissued by PMLR on 04 October 2026.

Related Material