Bayesian Optimization with Unknown Constraints

Michael Gelbart Harvard University, Jasper Snoek Harvard University, Ryan Adams Harvard
Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, PMLR R12:46-55, 2014.

Abstract

Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general case that noise may be present in the constraint functions, and the objective and constraints may be evaluated independently. We provide motivating practical examples, and present a general framework to solve such problems. We demonstrate the effectiveness of our approach on optimizing the performance of online latent Dirichlet allocation subject to topic sparsity constraints, tuning a neural network given test-time memory constraints, and optimizing Hamiltonian Monte Carlo to achieve maximal effectiveness in a fixed time, subject to passing standard convergence diagnostics.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR12-university14b, title = {{B}ayesian Optimization with Unknown Constraints}, author = {University, Michael Gelbart Harvard and University, Jasper Snoek Harvard and Harvard, Ryan Adams}, booktitle = {Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence}, pages = {46--55}, year = {2014}, editor = {Zhang, Nevin L. and Tian, Jin}, volume = {R12}, series = {Proceedings of Machine Learning Research}, month = {23--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r12/main/assets/university14b/university14b.pdf}, url = {https://proceedings.mlr.press/r12/university14b.html}, abstract = {Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general case that noise may be present in the constraint functions, and the objective and constraints may be evaluated independently. We provide motivating practical examples, and present a general framework to solve such problems. We demonstrate the effectiveness of our approach on optimizing the performance of online latent Dirichlet allocation subject to topic sparsity constraints, tuning a neural network given test-time memory constraints, and optimizing Hamiltonian Monte Carlo to achieve maximal effectiveness in a fixed time, subject to passing standard convergence diagnostics.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Bayesian Optimization with Unknown Constraints %A Michael Gelbart Harvard University %A Jasper Snoek Harvard University %A Ryan Adams Harvard %B Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2014 %E Nevin L. Zhang %E Jin Tian %F pmlr-vR12-university14b %I PMLR %P 46--55 %U https://proceedings.mlr.press/r12/university14b.html %V R12 %X Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general case that noise may be present in the constraint functions, and the objective and constraints may be evaluated independently. We provide motivating practical examples, and present a general framework to solve such problems. We demonstrate the effectiveness of our approach on optimizing the performance of online latent Dirichlet allocation subject to topic sparsity constraints, tuning a neural network given test-time memory constraints, and optimizing Hamiltonian Monte Carlo to achieve maximal effectiveness in a fixed time, subject to passing standard convergence diagnostics. %Z Reissued by PMLR on 04 October 2026.
APA
University, M.G.H., University, J.S.H. & Harvard, R.A.. (2014). Bayesian Optimization with Unknown Constraints. Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R12:46-55 Available from https://proceedings.mlr.press/r12/university14b.html. Reissued by PMLR on 04 October 2026.

Related Material