Random Projection in Neural Episodic Control

Daichi Nishio, Satoshi Yamane
Proceedings of The Eleventh Asian Conference on Machine Learning, PMLR 101:1-15, 2019.

Abstract

End-to-end deep reinforcement learning has enabled agents to learn with little preprocessing by humans. However, it is still difficult to learn stably and efficiently because the learning method usually uses a nonlinear function approximation. Neural Episodic Control (NEC), which has been proposed in order to improve sample efficiency, is able to learn stably by estimating action values using a non-parametric method. In this paper, we propose an architecture that incorporates random projection into NEC to train with more stability. In addition, we verify the effectiveness of our architecture by Atari’s five games. The main idea is to reduce the number of parameters that have to learn by replacing neural networks with random projection in order to reduce dimensions while keeping the learning end-to-end.

Cite this Paper

BibTeX
@InProceedings{pmlr-v101-nishio19a, title = {Random Projection in Neural Episodic Control}, author = {Nishio, Daichi and Yamane, Satoshi}, booktitle = {Proceedings of The Eleventh Asian Conference on Machine Learning}, pages = {1--15}, year = {2019}, editor = {Lee, Wee Sun and Suzuki, Taiji}, volume = {101}, series = {Proceedings of Machine Learning Research}, month = {17--19 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v101/nishio19a/nishio19a.pdf}, url = {https://proceedings.mlr.press/v101/nishio19a.html}, abstract = {End-to-end deep reinforcement learning has enabled agents to learn with little preprocessing by humans. However, it is still difficult to learn stably and efficiently because the learning method usually uses a nonlinear function approximation. Neural Episodic Control (NEC), which has been proposed in order to improve sample efficiency, is able to learn stably by estimating action values using a non-parametric method. In this paper, we propose an architecture that incorporates random projection into NEC to train with more stability. In addition, we verify the effectiveness of our architecture by Atari’s five games. The main idea is to reduce the number of parameters that have to learn by replacing neural networks with random projection in order to reduce dimensions while keeping the learning end-to-end.} }
Endnote
%0 Conference Paper %T Random Projection in Neural Episodic Control %A Daichi Nishio %A Satoshi Yamane %B Proceedings of The Eleventh Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Wee Sun Lee %E Taiji Suzuki %F pmlr-v101-nishio19a %I PMLR %P 1--15 %U https://proceedings.mlr.press/v101/nishio19a.html %V 101 %X End-to-end deep reinforcement learning has enabled agents to learn with little preprocessing by humans. However, it is still difficult to learn stably and efficiently because the learning method usually uses a nonlinear function approximation. Neural Episodic Control (NEC), which has been proposed in order to improve sample efficiency, is able to learn stably by estimating action values using a non-parametric method. In this paper, we propose an architecture that incorporates random projection into NEC to train with more stability. In addition, we verify the effectiveness of our architecture by Atari’s five games. The main idea is to reduce the number of parameters that have to learn by replacing neural networks with random projection in order to reduce dimensions while keeping the learning end-to-end.
APA
Nishio, D. & Yamane, S.. (2019). Random Projection in Neural Episodic Control. Proceedings of The Eleventh Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 101:1-15 Available from https://proceedings.mlr.press/v101/nishio19a.html.

Related Material