Sharing Less is More: Lifelong Learning in Deep Networks with Selective Layer Transfer

Seungwon Lee, Sima Behpour, Eric Eaton
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:6065-6075, 2021.

Abstract

Effective lifelong learning across diverse tasks requires the transfer of diverse knowledge, yet transferring irrelevant knowledge may lead to interference and catastrophic forgetting. In deep networks, transferring the appropriate granularity of knowledge is as important as the transfer mechanism, and must be driven by the relationships among tasks. We first show that the lifelong learning performance of several current deep learning architectures can be significantly improved by transfer at the appropriate layers. We then develop an expectation-maximization (EM) method to automatically select the appropriate transfer configuration and optimize the task network weights. This EM-based selective transfer is highly effective, balancing transfer performance on all tasks with avoiding catastrophic forgetting, as demonstrated on three algorithms in several lifelong object classification scenarios.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-lee21a, title = {Sharing Less is More: Lifelong Learning in Deep Networks with Selective Layer Transfer}, author = {Lee, Seungwon and Behpour, Sima and Eaton, Eric}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {6065--6075}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/lee21a/lee21a.pdf}, url = {https://proceedings.mlr.press/v139/lee21a.html}, abstract = {Effective lifelong learning across diverse tasks requires the transfer of diverse knowledge, yet transferring irrelevant knowledge may lead to interference and catastrophic forgetting. In deep networks, transferring the appropriate granularity of knowledge is as important as the transfer mechanism, and must be driven by the relationships among tasks. We first show that the lifelong learning performance of several current deep learning architectures can be significantly improved by transfer at the appropriate layers. We then develop an expectation-maximization (EM) method to automatically select the appropriate transfer configuration and optimize the task network weights. This EM-based selective transfer is highly effective, balancing transfer performance on all tasks with avoiding catastrophic forgetting, as demonstrated on three algorithms in several lifelong object classification scenarios.} }
Endnote
%0 Conference Paper %T Sharing Less is More: Lifelong Learning in Deep Networks with Selective Layer Transfer %A Seungwon Lee %A Sima Behpour %A Eric Eaton %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-lee21a %I PMLR %P 6065--6075 %U https://proceedings.mlr.press/v139/lee21a.html %V 139 %X Effective lifelong learning across diverse tasks requires the transfer of diverse knowledge, yet transferring irrelevant knowledge may lead to interference and catastrophic forgetting. In deep networks, transferring the appropriate granularity of knowledge is as important as the transfer mechanism, and must be driven by the relationships among tasks. We first show that the lifelong learning performance of several current deep learning architectures can be significantly improved by transfer at the appropriate layers. We then develop an expectation-maximization (EM) method to automatically select the appropriate transfer configuration and optimize the task network weights. This EM-based selective transfer is highly effective, balancing transfer performance on all tasks with avoiding catastrophic forgetting, as demonstrated on three algorithms in several lifelong object classification scenarios.
APA
Lee, S., Behpour, S. & Eaton, E.. (2021). Sharing Less is More: Lifelong Learning in Deep Networks with Selective Layer Transfer. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:6065-6075 Available from https://proceedings.mlr.press/v139/lee21a.html.

Related Material