Online Parameter-Free Learning of Multiple Low Variance Tasks

Giulia Denevi, Massimiliano Pontil, Dimitrios Stamos
Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), PMLR 124:889-898, 2020.

Abstract

We propose a method to learn a common bias vector for a growing sequence of low-variance tasks. Unlike state-of-the-art approaches, our method does not require tuning any hyper-parameter. Our approach is presented in the non-statistical setting and can be of two variants. The “aggressive” one updates the bias after each datapoint, the “lazy” one updates the bias only at the end of each task. We derive an across-tasks regret bound for the method. When compared to state-of-the-art approaches, the aggressive variant returns faster rates, the lazy one recovers standard rates, but with no need of tuning hyper-parameters. We then adapt the methods to the statistical setting: the aggressive variant becomes a multi-task learning method, the lazy one a meta-learning method. Experiments confirm the effectiveness of our methods in practice.

Cite this Paper

BibTeX
@InProceedings{pmlr-v124-denevi20a, title = {Online Parameter-Free Learning of Multiple Low Variance Tasks}, author = {Denevi, Giulia and Pontil, Massimiliano and Stamos, Dimitrios}, booktitle = {Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)}, pages = {889--898}, year = {2020}, editor = {Peters, Jonas and Sontag, David}, volume = {124}, series = {Proceedings of Machine Learning Research}, month = {03--06 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v124/denevi20a/denevi20a.pdf}, url = {https://proceedings.mlr.press/v124/denevi20a.html}, abstract = {We propose a method to learn a common bias vector for a growing sequence of low-variance tasks. Unlike state-of-the-art approaches, our method does not require tuning any hyper-parameter. Our approach is presented in the non-statistical setting and can be of two variants. The “aggressive” one updates the bias after each datapoint, the “lazy” one updates the bias only at the end of each task. We derive an across-tasks regret bound for the method. When compared to state-of-the-art approaches, the aggressive variant returns faster rates, the lazy one recovers standard rates, but with no need of tuning hyper-parameters. We then adapt the methods to the statistical setting: the aggressive variant becomes a multi-task learning method, the lazy one a meta-learning method. Experiments confirm the effectiveness of our methods in practice.} }
Endnote
%0 Conference Paper %T Online Parameter-Free Learning of Multiple Low Variance Tasks %A Giulia Denevi %A Massimiliano Pontil %A Dimitrios Stamos %B Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI) %C Proceedings of Machine Learning Research %D 2020 %E Jonas Peters %E David Sontag %F pmlr-v124-denevi20a %I PMLR %P 889--898 %U https://proceedings.mlr.press/v124/denevi20a.html %V 124 %X We propose a method to learn a common bias vector for a growing sequence of low-variance tasks. Unlike state-of-the-art approaches, our method does not require tuning any hyper-parameter. Our approach is presented in the non-statistical setting and can be of two variants. The “aggressive” one updates the bias after each datapoint, the “lazy” one updates the bias only at the end of each task. We derive an across-tasks regret bound for the method. When compared to state-of-the-art approaches, the aggressive variant returns faster rates, the lazy one recovers standard rates, but with no need of tuning hyper-parameters. We then adapt the methods to the statistical setting: the aggressive variant becomes a multi-task learning method, the lazy one a meta-learning method. Experiments confirm the effectiveness of our methods in practice.
APA
Denevi, G., Pontil, M. & Stamos, D.. (2020). Online Parameter-Free Learning of Multiple Low Variance Tasks. Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), in Proceedings of Machine Learning Research 124:889-898 Available from https://proceedings.mlr.press/v124/denevi20a.html.

Related Material