Neural Episodic Control

Alexander Pritzel, Benigno Uria, Sriram Srinivasan, Adrià Puigdomènech Badia, Oriol Vinyals, Demis Hassabis, Daan Wierstra, Charles Blundell
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:2827-2836, 2017.

Abstract

Deep reinforcement learning methods attain super-human performance in a wide range of environments. Such methods are grossly inefficient, often taking orders of magnitudes more data than humans to achieve reasonable performance. We propose Neural Episodic Control: a deep reinforcement learning agent that is able to rapidly assimilate new experiences and act upon them. Our agent uses a semi-tabular representation of the value function: a buffer of past experience containing slowly changing state representations and rapidly updated estimates of the value function. We show across a wide range of environments that our agent learns significantly faster than other state-of-the-art, general purpose deep reinforcement learning agents.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-pritzel17a, title = {Neural Episodic Control}, author = {Alexander Pritzel and Benigno Uria and Sriram Srinivasan and Adri{\`a} Puigdom{\`e}nech Badia and Oriol Vinyals and Demis Hassabis and Daan Wierstra and Charles Blundell}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {2827--2836}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/pritzel17a/pritzel17a.pdf}, url = {https://proceedings.mlr.press/v70/pritzel17a.html}, abstract = {Deep reinforcement learning methods attain super-human performance in a wide range of environments. Such methods are grossly inefficient, often taking orders of magnitudes more data than humans to achieve reasonable performance. We propose Neural Episodic Control: a deep reinforcement learning agent that is able to rapidly assimilate new experiences and act upon them. Our agent uses a semi-tabular representation of the value function: a buffer of past experience containing slowly changing state representations and rapidly updated estimates of the value function. We show across a wide range of environments that our agent learns significantly faster than other state-of-the-art, general purpose deep reinforcement learning agents.} }
Endnote
%0 Conference Paper %T Neural Episodic Control %A Alexander Pritzel %A Benigno Uria %A Sriram Srinivasan %A Adrià Puigdomènech Badia %A Oriol Vinyals %A Demis Hassabis %A Daan Wierstra %A Charles Blundell %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-pritzel17a %I PMLR %P 2827--2836 %U https://proceedings.mlr.press/v70/pritzel17a.html %V 70 %X Deep reinforcement learning methods attain super-human performance in a wide range of environments. Such methods are grossly inefficient, often taking orders of magnitudes more data than humans to achieve reasonable performance. We propose Neural Episodic Control: a deep reinforcement learning agent that is able to rapidly assimilate new experiences and act upon them. Our agent uses a semi-tabular representation of the value function: a buffer of past experience containing slowly changing state representations and rapidly updated estimates of the value function. We show across a wide range of environments that our agent learns significantly faster than other state-of-the-art, general purpose deep reinforcement learning agents.
APA
Pritzel, A., Uria, B., Srinivasan, S., Badia, A.P., Vinyals, O., Hassabis, D., Wierstra, D. & Blundell, C.. (2017). Neural Episodic Control. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:2827-2836 Available from https://proceedings.mlr.press/v70/pritzel17a.html.

Related Material