Distributionally Robust Bayesian Optimization

Johannes Kirschner, Ilija Bogunovic, Stefanie Jegelka, Andreas Krause
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR 108:2174-2184, 2020.

Abstract

Robustness to distributional shift is one of the key challenges of contemporary machine learning. Attaining such robustness is the goal of distributionally robust optimization, which seeks a solution to an optimization problem that is worst-case robust under a specified distributional shift of an uncontrolled covariate. In this paper, we study such a problem when the distributional shift is measured via the maximum mean discrepancy (MMD). For the setting of zeroth-order, noisy optimization, we present a novel distributionally robust Bayesian optimization algorithm (DRBO). Our algorithm provably obtains sub-linear robust regret in various settings that differ in how the uncertain covariate is observed. We demonstrate the robust performance of our method on both synthetic and real-world benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v108-kirschner20a, title = {Distributionally Robust Bayesian Optimization}, author = {Kirschner, Johannes and Bogunovic, Ilija and Jegelka, Stefanie and Krause, Andreas}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {2174--2184}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/kirschner20a/kirschner20a.pdf}, url = {https://proceedings.mlr.press/v108/kirschner20a.html}, abstract = {Robustness to distributional shift is one of the key challenges of contemporary machine learning. Attaining such robustness is the goal of distributionally robust optimization, which seeks a solution to an optimization problem that is worst-case robust under a specified distributional shift of an uncontrolled covariate. In this paper, we study such a problem when the distributional shift is measured via the maximum mean discrepancy (MMD). For the setting of zeroth-order, noisy optimization, we present a novel distributionally robust Bayesian optimization algorithm (DRBO). Our algorithm provably obtains sub-linear robust regret in various settings that differ in how the uncertain covariate is observed. We demonstrate the robust performance of our method on both synthetic and real-world benchmarks.} }
Endnote
%0 Conference Paper %T Distributionally Robust Bayesian Optimization %A Johannes Kirschner %A Ilija Bogunovic %A Stefanie Jegelka %A Andreas Krause %B Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2020 %E Silvia Chiappa %E Roberto Calandra %F pmlr-v108-kirschner20a %I PMLR %P 2174--2184 %U https://proceedings.mlr.press/v108/kirschner20a.html %V 108 %X Robustness to distributional shift is one of the key challenges of contemporary machine learning. Attaining such robustness is the goal of distributionally robust optimization, which seeks a solution to an optimization problem that is worst-case robust under a specified distributional shift of an uncontrolled covariate. In this paper, we study such a problem when the distributional shift is measured via the maximum mean discrepancy (MMD). For the setting of zeroth-order, noisy optimization, we present a novel distributionally robust Bayesian optimization algorithm (DRBO). Our algorithm provably obtains sub-linear robust regret in various settings that differ in how the uncertain covariate is observed. We demonstrate the robust performance of our method on both synthetic and real-world benchmarks.
APA
Kirschner, J., Bogunovic, I., Jegelka, S. & Krause, A.. (2020). Distributionally Robust Bayesian Optimization. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 108:2174-2184 Available from https://proceedings.mlr.press/v108/kirschner20a.html.

Related Material