Blackbox optimization of unimodal functions

A. Cutkosky, A. Das, W. Kong, C. Lee, R. Sen
Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, PMLR 216:476-484, 2023.

Abstract

We provide an intuitive new algorithm for blackbox stochastic optimization of unimodal functions, a function class that we observe empirically can capture hyperparameter-tuning loss surfaces. Our method’s convergence guarantee automatically adapts to Lipschitz constants and other problem difficulty parameters, recovering and extending prior results. We complement our theoretical development with experimental validation on hyperparameter tuning tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v216-cutkosky23a, title = {Blackbox optimization of unimodal functions}, author = {Cutkosky, A. and Das, A. and Kong, W. and Lee, C. and Sen, R.}, booktitle = {Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence}, pages = {476--484}, year = {2023}, editor = {Evans, Robin J. and Shpitser, Ilya}, volume = {216}, series = {Proceedings of Machine Learning Research}, month = {31 Jul--04 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v216/cutkosky23a/cutkosky23a.pdf}, url = {https://proceedings.mlr.press/v216/cutkosky23a.html}, abstract = {We provide an intuitive new algorithm for blackbox stochastic optimization of unimodal functions, a function class that we observe empirically can capture hyperparameter-tuning loss surfaces. Our method’s convergence guarantee automatically adapts to Lipschitz constants and other problem difficulty parameters, recovering and extending prior results. We complement our theoretical development with experimental validation on hyperparameter tuning tasks.} }
Endnote
%0 Conference Paper %T Blackbox optimization of unimodal functions %A A. Cutkosky %A A. Das %A W. Kong %A C. Lee %A R. Sen %B Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2023 %E Robin J. Evans %E Ilya Shpitser %F pmlr-v216-cutkosky23a %I PMLR %P 476--484 %U https://proceedings.mlr.press/v216/cutkosky23a.html %V 216 %X We provide an intuitive new algorithm for blackbox stochastic optimization of unimodal functions, a function class that we observe empirically can capture hyperparameter-tuning loss surfaces. Our method’s convergence guarantee automatically adapts to Lipschitz constants and other problem difficulty parameters, recovering and extending prior results. We complement our theoretical development with experimental validation on hyperparameter tuning tasks.
APA
Cutkosky, A., Das, A., Kong, W., Lee, C. & Sen, R.. (2023). Blackbox optimization of unimodal functions. Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 216:476-484 Available from https://proceedings.mlr.press/v216/cutkosky23a.html.

Related Material