Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Blackbox Optimization

Xingyou Song, Sagi Perel, Chansoo Lee, Greg Kochanski, Daniel Golovin
Proceedings of the First International Conference on Automated Machine Learning, PMLR 188:8/1-17, 2022.

Abstract

Vizier is the de-facto blackbox optimization service across Google, having optimized some of Google’s largest products and research efforts. To operate at the scale of tuning thousands of users’ critical systems, Vizier solved key design challenges in providing multiple different features, while remaining fully fault-tolerant. In this paper, we introduce Open Source (OSS) Vizier, a Python-based interface for blackbox optimization and research, based on the Google-internal Vizier infrastructure and framework. OSS Vizier provides an API capable of defining and solving a wide variety of optimization problems, including multi-metric, early stopping, transfer learning, and conditional search. Furthermore, it is designed to be a distributed system that assures reliability, and allows multiple parallel evaluations of the user’s objective function. The flexible RPC-based infrastructure allows users to access OSS Vizier from binaries written in any language. OSS Vizier also provides a back-end (”Pythia”) API that gives algorithm authors a way to interface new algorithms with the core Vizier system. OSS Vizier is available at \url{https://github.com/google/vizier}.

Cite this Paper

BibTeX
@InProceedings{pmlr-v188-song22a, title = {Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Blackbox Optimization}, author = {Song, Xingyou and Perel, Sagi and Lee, Chansoo and Kochanski, Greg and Golovin, Daniel}, booktitle = {Proceedings of the First International Conference on Automated Machine Learning}, pages = {8/1--17}, year = {2022}, editor = {Guyon, Isabelle and Lindauer, Marius and van der Schaar, Mihaela and Hutter, Frank and Garnett, Roman}, volume = {188}, series = {Proceedings of Machine Learning Research}, month = {25--27 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v188/song22a/song22a.pdf}, url = {https://proceedings.mlr.press/v188/song22a.html}, abstract = {Vizier is the de-facto blackbox optimization service across Google, having optimized some of Google’s largest products and research efforts. To operate at the scale of tuning thousands of users’ critical systems, Vizier solved key design challenges in providing multiple different features, while remaining fully fault-tolerant. In this paper, we introduce Open Source (OSS) Vizier, a Python-based interface for blackbox optimization and research, based on the Google-internal Vizier infrastructure and framework. OSS Vizier provides an API capable of defining and solving a wide variety of optimization problems, including multi-metric, early stopping, transfer learning, and conditional search. Furthermore, it is designed to be a distributed system that assures reliability, and allows multiple parallel evaluations of the user’s objective function. The flexible RPC-based infrastructure allows users to access OSS Vizier from binaries written in any language. OSS Vizier also provides a back-end (”Pythia”) API that gives algorithm authors a way to interface new algorithms with the core Vizier system. OSS Vizier is available at \url{https://github.com/google/vizier}.} }
Endnote
%0 Conference Paper %T Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Blackbox Optimization %A Xingyou Song %A Sagi Perel %A Chansoo Lee %A Greg Kochanski %A Daniel Golovin %B Proceedings of the First International Conference on Automated Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Isabelle Guyon %E Marius Lindauer %E Mihaela van der Schaar %E Frank Hutter %E Roman Garnett %F pmlr-v188-song22a %I PMLR %P 8/1--17 %U https://proceedings.mlr.press/v188/song22a.html %V 188 %X Vizier is the de-facto blackbox optimization service across Google, having optimized some of Google’s largest products and research efforts. To operate at the scale of tuning thousands of users’ critical systems, Vizier solved key design challenges in providing multiple different features, while remaining fully fault-tolerant. In this paper, we introduce Open Source (OSS) Vizier, a Python-based interface for blackbox optimization and research, based on the Google-internal Vizier infrastructure and framework. OSS Vizier provides an API capable of defining and solving a wide variety of optimization problems, including multi-metric, early stopping, transfer learning, and conditional search. Furthermore, it is designed to be a distributed system that assures reliability, and allows multiple parallel evaluations of the user’s objective function. The flexible RPC-based infrastructure allows users to access OSS Vizier from binaries written in any language. OSS Vizier also provides a back-end (”Pythia”) API that gives algorithm authors a way to interface new algorithms with the core Vizier system. OSS Vizier is available at \url{https://github.com/google/vizier}.
APA
Song, X., Perel, S., Lee, C., Kochanski, G. & Golovin, D.. (2022). Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Blackbox Optimization. Proceedings of the First International Conference on Automated Machine Learning, in Proceedings of Machine Learning Research 188:8/1-17 Available from https://proceedings.mlr.press/v188/song22a.html.

Related Material