Been There, Done That: Meta-Learning with Episodic Recall

Samuel Ritter, Jane Wang, Zeb Kurth-Nelson, Siddhant Jayakumar, Charles Blundell, Razvan Pascanu, Matthew Botvinick
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:4354-4363, 2018.

Abstract

Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur {–} as they do in natural environments {–} meta-learning agents must explore again instead of immediately exploiting previously discovered solutions. We propose a formalism for generating open-ended yet repetitious environments, then develop a meta-learning architecture for solving these environments. This architecture melds the standard LSTM working memory with a differentiable neural episodic memory. We explore the capabilities of agents with this episodic LSTM in five meta-learning environments with reoccurring tasks, ranging from bandits to navigation and stochastic sequential decision problems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-ritter18a, title = {Been There, Done That: Meta-Learning with Episodic Recall}, author = {Ritter, Samuel and Wang, Jane and Kurth-Nelson, Zeb and Jayakumar, Siddhant and Blundell, Charles and Pascanu, Razvan and Botvinick, Matthew}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {4354--4363}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/ritter18a/ritter18a.pdf}, url = {https://proceedings.mlr.press/v80/ritter18a.html}, abstract = {Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur {–} as they do in natural environments {–} meta-learning agents must explore again instead of immediately exploiting previously discovered solutions. We propose a formalism for generating open-ended yet repetitious environments, then develop a meta-learning architecture for solving these environments. This architecture melds the standard LSTM working memory with a differentiable neural episodic memory. We explore the capabilities of agents with this episodic LSTM in five meta-learning environments with reoccurring tasks, ranging from bandits to navigation and stochastic sequential decision problems.} }
Endnote
%0 Conference Paper %T Been There, Done That: Meta-Learning with Episodic Recall %A Samuel Ritter %A Jane Wang %A Zeb Kurth-Nelson %A Siddhant Jayakumar %A Charles Blundell %A Razvan Pascanu %A Matthew Botvinick %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-ritter18a %I PMLR %P 4354--4363 %U https://proceedings.mlr.press/v80/ritter18a.html %V 80 %X Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur {–} as they do in natural environments {–} meta-learning agents must explore again instead of immediately exploiting previously discovered solutions. We propose a formalism for generating open-ended yet repetitious environments, then develop a meta-learning architecture for solving these environments. This architecture melds the standard LSTM working memory with a differentiable neural episodic memory. We explore the capabilities of agents with this episodic LSTM in five meta-learning environments with reoccurring tasks, ranging from bandits to navigation and stochastic sequential decision problems.
APA
Ritter, S., Wang, J., Kurth-Nelson, Z., Jayakumar, S., Blundell, C., Pascanu, R. & Botvinick, M.. (2018). Been There, Done That: Meta-Learning with Episodic Recall. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:4354-4363 Available from https://proceedings.mlr.press/v80/ritter18a.html.

Related Material