Incremental Learning-to-Learn with Statistical Guarantees

Giulia Denevi, Carlo Ciliberto, Dimitris Stamos, Massimiliano Pontil
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:456-465, 2018.

Abstract

In learning-to-learn the goal is to infer a learning algorithm that works well on a class of tasks sampled from an unknown meta- distribution. In contrast to previous work on batch learning-to-learn, we consider a scenario where tasks are presented sequentially and the algorithm needs to adapt incrementally to im- prove its performance on future tasks. Key to this setting is for the algorithm to rapidly in- corporate new observations into the model as they arrive, without keeping them in memory. We focus on the case where the underlying al- gorithm is Ridge Regression parametrised by a symmetric positive semidefinite matrix. We propose to learn this matrix by applying a stochastic strategy to minimize the empirical error incurred by Ridge Regression on future tasks sampled from the meta-distribution. We study the statistical properties of the proposed algorithm and prove non-asymptotic bounds on its excess transfer risk, that is, the gener- alization performance on new tasks from the same meta-distribution. We compare our on- line learning-to-learn approach with a state-of- the-art batch method, both theoretically and empirically.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-denevi18a, title = {Incremental Learning-to-Learn with Statistical Guarantees}, author = {Denevi, Giulia and Ciliberto, Carlo and Stamos, Dimitris and Pontil, Massimiliano}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {456--465}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/denevi18a/denevi18a.pdf}, url = {https://proceedings.mlr.press/r16/denevi18a.html}, abstract = {In learning-to-learn the goal is to infer a learning algorithm that works well on a class of tasks sampled from an unknown meta- distribution. In contrast to previous work on batch learning-to-learn, we consider a scenario where tasks are presented sequentially and the algorithm needs to adapt incrementally to im- prove its performance on future tasks. Key to this setting is for the algorithm to rapidly in- corporate new observations into the model as they arrive, without keeping them in memory. We focus on the case where the underlying al- gorithm is Ridge Regression parametrised by a symmetric positive semidefinite matrix. We propose to learn this matrix by applying a stochastic strategy to minimize the empirical error incurred by Ridge Regression on future tasks sampled from the meta-distribution. We study the statistical properties of the proposed algorithm and prove non-asymptotic bounds on its excess transfer risk, that is, the gener- alization performance on new tasks from the same meta-distribution. We compare our on- line learning-to-learn approach with a state-of- the-art batch method, both theoretically and empirically.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Incremental Learning-to-Learn with Statistical Guarantees %A Giulia Denevi %A Carlo Ciliberto %A Dimitris Stamos %A Massimiliano Pontil %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-denevi18a %I PMLR %P 456--465 %U https://proceedings.mlr.press/r16/denevi18a.html %V R16 %X In learning-to-learn the goal is to infer a learning algorithm that works well on a class of tasks sampled from an unknown meta- distribution. In contrast to previous work on batch learning-to-learn, we consider a scenario where tasks are presented sequentially and the algorithm needs to adapt incrementally to im- prove its performance on future tasks. Key to this setting is for the algorithm to rapidly in- corporate new observations into the model as they arrive, without keeping them in memory. We focus on the case where the underlying al- gorithm is Ridge Regression parametrised by a symmetric positive semidefinite matrix. We propose to learn this matrix by applying a stochastic strategy to minimize the empirical error incurred by Ridge Regression on future tasks sampled from the meta-distribution. We study the statistical properties of the proposed algorithm and prove non-asymptotic bounds on its excess transfer risk, that is, the gener- alization performance on new tasks from the same meta-distribution. We compare our on- line learning-to-learn approach with a state-of- the-art batch method, both theoretically and empirically. %Z Reissued by PMLR on 04 October 2026.
APA
Denevi, G., Ciliberto, C., Stamos, D. & Pontil, M.. (2018). Incremental Learning-to-Learn with Statistical Guarantees. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:456-465 Available from https://proceedings.mlr.press/r16/denevi18a.html. Reissued by PMLR on 04 October 2026.

Related Material