Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes

Yan Zhang, Xuefeng Liu, Sipeng Chen, Sascha Ranftl, Chong Liu, Shibo Li
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:157148-157179, 2026.

Abstract

Standard Bayesian Optimization (BO) assumes uniform smoothness across the search space—an assumption violated in multi-regime problems such as molecular conformation search through distinct energy basins or drug discovery across heterogeneous molecular scaffolds. A single GP either oversmooths sharp transitions or hallucinates noise in smooth regions, yielding miscalibrated uncertainty. We propose RAMBO, a Dirichlet Process Mixture of Gaussian Processes that automatically discovers latent regimes during optimization, each modeled by an independent GP with locally-optimized hyperparameters. We derive collapsed Gibbs sampling that analytically marginalizes latent functions for efficient inference, and introduce adaptive concentration parameter scheduling for coarse-to-fine regime discovery. Our acquisition functions decompose uncertainty into intra-regime and inter-regime components. Experiments on synthetic benchmarks and real-world applications—including molecular conformer optimization, virtual screening for drug discovery, and fusion reactor design—demonstrate consistent improvements over state-of-the-art baselines on multi-regime objectives. Code is available at https://github.com/AnthonyZhangYan/RAMBO.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-zhang26dn, title = {Regime-Adaptive {B}ayesian Optimization via {D}irichlet Process Mixtures of {G}aussian Processes}, author = {Zhang, Yan and Liu, Xuefeng and Chen, Sipeng and Ranftl, Sascha and Liu, Chong and Li, Shibo}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {157148--157179}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/zhang26dn/zhang26dn.pdf}, url = {https://proceedings.mlr.press/v306/zhang26dn.html}, abstract = {Standard Bayesian Optimization (BO) assumes uniform smoothness across the search space—an assumption violated in multi-regime problems such as molecular conformation search through distinct energy basins or drug discovery across heterogeneous molecular scaffolds. A single GP either oversmooths sharp transitions or hallucinates noise in smooth regions, yielding miscalibrated uncertainty. We propose RAMBO, a Dirichlet Process Mixture of Gaussian Processes that automatically discovers latent regimes during optimization, each modeled by an independent GP with locally-optimized hyperparameters. We derive collapsed Gibbs sampling that analytically marginalizes latent functions for efficient inference, and introduce adaptive concentration parameter scheduling for coarse-to-fine regime discovery. Our acquisition functions decompose uncertainty into intra-regime and inter-regime components. Experiments on synthetic benchmarks and real-world applications—including molecular conformer optimization, virtual screening for drug discovery, and fusion reactor design—demonstrate consistent improvements over state-of-the-art baselines on multi-regime objectives. Code is available at https://github.com/AnthonyZhangYan/RAMBO.} }
Endnote
%0 Conference Paper %T Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes %A Yan Zhang %A Xuefeng Liu %A Sipeng Chen %A Sascha Ranftl %A Chong Liu %A Shibo Li %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-zhang26dn %I PMLR %P 157148--157179 %U https://proceedings.mlr.press/v306/zhang26dn.html %V 306 %X Standard Bayesian Optimization (BO) assumes uniform smoothness across the search space—an assumption violated in multi-regime problems such as molecular conformation search through distinct energy basins or drug discovery across heterogeneous molecular scaffolds. A single GP either oversmooths sharp transitions or hallucinates noise in smooth regions, yielding miscalibrated uncertainty. We propose RAMBO, a Dirichlet Process Mixture of Gaussian Processes that automatically discovers latent regimes during optimization, each modeled by an independent GP with locally-optimized hyperparameters. We derive collapsed Gibbs sampling that analytically marginalizes latent functions for efficient inference, and introduce adaptive concentration parameter scheduling for coarse-to-fine regime discovery. Our acquisition functions decompose uncertainty into intra-regime and inter-regime components. Experiments on synthetic benchmarks and real-world applications—including molecular conformer optimization, virtual screening for drug discovery, and fusion reactor design—demonstrate consistent improvements over state-of-the-art baselines on multi-regime objectives. Code is available at https://github.com/AnthonyZhangYan/RAMBO.
APA
Zhang, Y., Liu, X., Chen, S., Ranftl, S., Liu, C. & Li, S.. (2026). Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:157148-157179 Available from https://proceedings.mlr.press/v306/zhang26dn.html.

Related Material