ChaCha for Online AutoML

Qingyun Wu, Chi Wang, John Langford, Paul Mineiro, Marco Rossi
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:11263-11273, 2021.

Abstract

We propose the ChaCha (Champion-Challengers) algorithm for making an online choice of hyperparameters in online learning settings. ChaCha handles the process of determining a champion and scheduling a set of ‘live’ challengers over time based on sample complexity bounds. It is guaranteed to have sublinear regret after the optimal configuration is added into consideration by an application-dependent oracle based on the champions. Empirically, we show that ChaCha provides good performance across a wide array of datasets when optimizing over featurization and hyperparameter decisions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-wu21d, title = {ChaCha for Online AutoML}, author = {Wu, Qingyun and Wang, Chi and Langford, John and Mineiro, Paul and Rossi, Marco}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {11263--11273}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/wu21d/wu21d.pdf}, url = {https://proceedings.mlr.press/v139/wu21d.html}, abstract = {We propose the ChaCha (Champion-Challengers) algorithm for making an online choice of hyperparameters in online learning settings. ChaCha handles the process of determining a champion and scheduling a set of ‘live’ challengers over time based on sample complexity bounds. It is guaranteed to have sublinear regret after the optimal configuration is added into consideration by an application-dependent oracle based on the champions. Empirically, we show that ChaCha provides good performance across a wide array of datasets when optimizing over featurization and hyperparameter decisions.} }
Endnote
%0 Conference Paper %T ChaCha for Online AutoML %A Qingyun Wu %A Chi Wang %A John Langford %A Paul Mineiro %A Marco Rossi %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-wu21d %I PMLR %P 11263--11273 %U https://proceedings.mlr.press/v139/wu21d.html %V 139 %X We propose the ChaCha (Champion-Challengers) algorithm for making an online choice of hyperparameters in online learning settings. ChaCha handles the process of determining a champion and scheduling a set of ‘live’ challengers over time based on sample complexity bounds. It is guaranteed to have sublinear regret after the optimal configuration is added into consideration by an application-dependent oracle based on the champions. Empirically, we show that ChaCha provides good performance across a wide array of datasets when optimizing over featurization and hyperparameter decisions.
APA
Wu, Q., Wang, C., Langford, J., Mineiro, P. & Rossi, M.. (2021). ChaCha for Online AutoML. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:11263-11273 Available from https://proceedings.mlr.press/v139/wu21d.html.

Related Material