ELLA: An Efficient Lifelong Learning Algorithm

Paul Ruvolo, Eric Eaton
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(1):507-515, 2013.

Abstract

The problem of learning multiple consecutive tasks, known as lifelong learning, is of great importance to the creation of intelligent, general-purpose, and flexible machines. In this paper, we develop a method for online multi-task learning in the lifelong learning setting. The proposed Efficient Lifelong Learning Algorithm (ELLA) maintains a sparsely shared basis for all task models, transfers knowledge from the basis to learn each new task, and refines the basis over time to maximize performance across all tasks. We show that ELLA has strong connections to both online dictionary learning for sparse coding and state-of-the-art batch multi-task learning methods, and provide robust theoretical performance guarantees. We show empirically that ELLA yields nearly identical performance to batch multi-task learning while learning tasks sequentially in three orders of magnitude (over 1,000x) less time.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-ruvolo13, title = {{ELLA}: An Efficient Lifelong Learning Algorithm}, author = {Ruvolo, Paul and Eaton, Eric}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {507--515}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {1}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/ruvolo13.pdf}, url = {https://proceedings.mlr.press/v28/ruvolo13.html}, abstract = {The problem of learning multiple consecutive tasks, known as lifelong learning, is of great importance to the creation of intelligent, general-purpose, and flexible machines. In this paper, we develop a method for online multi-task learning in the lifelong learning setting. The proposed Efficient Lifelong Learning Algorithm (ELLA) maintains a sparsely shared basis for all task models, transfers knowledge from the basis to learn each new task, and refines the basis over time to maximize performance across all tasks. We show that ELLA has strong connections to both online dictionary learning for sparse coding and state-of-the-art batch multi-task learning methods, and provide robust theoretical performance guarantees. We show empirically that ELLA yields nearly identical performance to batch multi-task learning while learning tasks sequentially in three orders of magnitude (over 1,000x) less time.} }
Endnote
%0 Conference Paper %T ELLA: An Efficient Lifelong Learning Algorithm %A Paul Ruvolo %A Eric Eaton %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-ruvolo13 %I PMLR %P 507--515 %U https://proceedings.mlr.press/v28/ruvolo13.html %V 28 %N 1 %X The problem of learning multiple consecutive tasks, known as lifelong learning, is of great importance to the creation of intelligent, general-purpose, and flexible machines. In this paper, we develop a method for online multi-task learning in the lifelong learning setting. The proposed Efficient Lifelong Learning Algorithm (ELLA) maintains a sparsely shared basis for all task models, transfers knowledge from the basis to learn each new task, and refines the basis over time to maximize performance across all tasks. We show that ELLA has strong connections to both online dictionary learning for sparse coding and state-of-the-art batch multi-task learning methods, and provide robust theoretical performance guarantees. We show empirically that ELLA yields nearly identical performance to batch multi-task learning while learning tasks sequentially in three orders of magnitude (over 1,000x) less time.
RIS
TY - CPAPER TI - ELLA: An Efficient Lifelong Learning Algorithm AU - Paul Ruvolo AU - Eric Eaton BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/02/13 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-ruvolo13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 1 SP - 507 EP - 515 L1 - http://proceedings.mlr.press/v28/ruvolo13.pdf UR - https://proceedings.mlr.press/v28/ruvolo13.html AB - The problem of learning multiple consecutive tasks, known as lifelong learning, is of great importance to the creation of intelligent, general-purpose, and flexible machines. In this paper, we develop a method for online multi-task learning in the lifelong learning setting. The proposed Efficient Lifelong Learning Algorithm (ELLA) maintains a sparsely shared basis for all task models, transfers knowledge from the basis to learn each new task, and refines the basis over time to maximize performance across all tasks. We show that ELLA has strong connections to both online dictionary learning for sparse coding and state-of-the-art batch multi-task learning methods, and provide robust theoretical performance guarantees. We show empirically that ELLA yields nearly identical performance to batch multi-task learning while learning tasks sequentially in three orders of magnitude (over 1,000x) less time. ER -
APA
Ruvolo, P. & Eaton, E.. (2013). ELLA: An Efficient Lifelong Learning Algorithm. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(1):507-515 Available from https://proceedings.mlr.press/v28/ruvolo13.html.

Related Material