Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints

Yunsheng Tian, Ane Zuniga, Xinwei Zhang, Johannes P. Dürholt, Payel Das, Jie Chen, Wojciech Matusik, Mina Konakovic Lukovic
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:48295-48320, 2024.

Abstract

Bayesian optimization has been successfully applied to optimize black-box functions where the number of evaluations is severely limited. However, in many real-world applications, it is hard or impossible to know in advance which designs are feasible due to some physical or system limitations. These issues lead to an even more challenging problem of optimizing an unknown function with unknown constraints. In this paper, we observe that in such scenarios optimal solution typically lies on the boundary between feasible and infeasible regions of the design space, making it considerably more difficult than that with interior optima. Inspired by this observation, we propose BE-CBO, a new Bayesian optimization method that efficiently explores the boundary between feasible and infeasible designs. To identify the boundary, we learn the constraints with an ensemble of neural networks that outperform the standard Gaussian Processes for capturing complex boundaries. Our method demonstrates superior performance against state-of-the-art methods through comprehensive experiments on synthetic and real-world benchmarks. Code available at: https://github.com/yunshengtian/BE-CBO

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-tian24g, title = {Boundary Exploration for {B}ayesian Optimization With Unknown Physical Constraints}, author = {Tian, Yunsheng and Zuniga, Ane and Zhang, Xinwei and D\"{u}rholt, Johannes P. and Das, Payel and Chen, Jie and Matusik, Wojciech and Konakovic Lukovic, Mina}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {48295--48320}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/tian24g/tian24g.pdf}, url = {https://proceedings.mlr.press/v235/tian24g.html}, abstract = {Bayesian optimization has been successfully applied to optimize black-box functions where the number of evaluations is severely limited. However, in many real-world applications, it is hard or impossible to know in advance which designs are feasible due to some physical or system limitations. These issues lead to an even more challenging problem of optimizing an unknown function with unknown constraints. In this paper, we observe that in such scenarios optimal solution typically lies on the boundary between feasible and infeasible regions of the design space, making it considerably more difficult than that with interior optima. Inspired by this observation, we propose BE-CBO, a new Bayesian optimization method that efficiently explores the boundary between feasible and infeasible designs. To identify the boundary, we learn the constraints with an ensemble of neural networks that outperform the standard Gaussian Processes for capturing complex boundaries. Our method demonstrates superior performance against state-of-the-art methods through comprehensive experiments on synthetic and real-world benchmarks. Code available at: https://github.com/yunshengtian/BE-CBO} }
Endnote
%0 Conference Paper %T Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints %A Yunsheng Tian %A Ane Zuniga %A Xinwei Zhang %A Johannes P. Dürholt %A Payel Das %A Jie Chen %A Wojciech Matusik %A Mina Konakovic Lukovic %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-tian24g %I PMLR %P 48295--48320 %U https://proceedings.mlr.press/v235/tian24g.html %V 235 %X Bayesian optimization has been successfully applied to optimize black-box functions where the number of evaluations is severely limited. However, in many real-world applications, it is hard or impossible to know in advance which designs are feasible due to some physical or system limitations. These issues lead to an even more challenging problem of optimizing an unknown function with unknown constraints. In this paper, we observe that in such scenarios optimal solution typically lies on the boundary between feasible and infeasible regions of the design space, making it considerably more difficult than that with interior optima. Inspired by this observation, we propose BE-CBO, a new Bayesian optimization method that efficiently explores the boundary between feasible and infeasible designs. To identify the boundary, we learn the constraints with an ensemble of neural networks that outperform the standard Gaussian Processes for capturing complex boundaries. Our method demonstrates superior performance against state-of-the-art methods through comprehensive experiments on synthetic and real-world benchmarks. Code available at: https://github.com/yunshengtian/BE-CBO
APA
Tian, Y., Zuniga, A., Zhang, X., Dürholt, J.P., Das, P., Chen, J., Matusik, W. & Konakovic Lukovic, M.. (2024). Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:48295-48320 Available from https://proceedings.mlr.press/v235/tian24g.html.

Related Material