Self-Activating Neural Ensembles for Continual Reinforcement Learning

Sam Powers, Eliot Xing, Abhinav Gupta
Proceedings of The 1st Conference on Lifelong Learning Agents, PMLR 199:683-704, 2022.

Abstract

The ability for an agent to continuously learn new skills without catastrophically forgetting existing knowledge is of critical importance for the development of generally intelligent agents. Most methods devised to address this problem depend heavily on well-defined task boundaries, and thus depend on human supervision. Our task-agnostic method, Self-Activating Neural Ensembles (SANE), uses a modular architecture designed to avoid catastrophic forgetting without making any such assumptions. At the beginning of each trajectory, a module in the SANE ensemble is activated to determine the agent’s next policy. During training, new modules are created as needed and only activated modules are updated to ensure that unused modules remain unchanged. This system enables our method to retain and leverage old skills, while growing and learning new ones. We demonstrate our approach on visually rich procedurally generated environments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v199-powers22a, title = {Self-Activating Neural Ensembles for Continual Reinforcement Learning}, author = {Powers, Sam and Xing, Eliot and Gupta, Abhinav}, booktitle = {Proceedings of The 1st Conference on Lifelong Learning Agents}, pages = {683--704}, year = {2022}, editor = {Chandar, Sarath and Pascanu, Razvan and Precup, Doina}, volume = {199}, series = {Proceedings of Machine Learning Research}, month = {22--24 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v199/powers22a/powers22a.pdf}, url = {https://proceedings.mlr.press/v199/powers22a.html}, abstract = {The ability for an agent to continuously learn new skills without catastrophically forgetting existing knowledge is of critical importance for the development of generally intelligent agents. Most methods devised to address this problem depend heavily on well-defined task boundaries, and thus depend on human supervision. Our task-agnostic method, Self-Activating Neural Ensembles (SANE), uses a modular architecture designed to avoid catastrophic forgetting without making any such assumptions. At the beginning of each trajectory, a module in the SANE ensemble is activated to determine the agent’s next policy. During training, new modules are created as needed and only activated modules are updated to ensure that unused modules remain unchanged. This system enables our method to retain and leverage old skills, while growing and learning new ones. We demonstrate our approach on visually rich procedurally generated environments.} }
Endnote
%0 Conference Paper %T Self-Activating Neural Ensembles for Continual Reinforcement Learning %A Sam Powers %A Eliot Xing %A Abhinav Gupta %B Proceedings of The 1st Conference on Lifelong Learning Agents %C Proceedings of Machine Learning Research %D 2022 %E Sarath Chandar %E Razvan Pascanu %E Doina Precup %F pmlr-v199-powers22a %I PMLR %P 683--704 %U https://proceedings.mlr.press/v199/powers22a.html %V 199 %X The ability for an agent to continuously learn new skills without catastrophically forgetting existing knowledge is of critical importance for the development of generally intelligent agents. Most methods devised to address this problem depend heavily on well-defined task boundaries, and thus depend on human supervision. Our task-agnostic method, Self-Activating Neural Ensembles (SANE), uses a modular architecture designed to avoid catastrophic forgetting without making any such assumptions. At the beginning of each trajectory, a module in the SANE ensemble is activated to determine the agent’s next policy. During training, new modules are created as needed and only activated modules are updated to ensure that unused modules remain unchanged. This system enables our method to retain and leverage old skills, while growing and learning new ones. We demonstrate our approach on visually rich procedurally generated environments.
APA
Powers, S., Xing, E. & Gupta, A.. (2022). Self-Activating Neural Ensembles for Continual Reinforcement Learning. Proceedings of The 1st Conference on Lifelong Learning Agents, in Proceedings of Machine Learning Research 199:683-704 Available from https://proceedings.mlr.press/v199/powers22a.html.

Related Material