Graying the black box: Understanding DQNs

Tom Zahavy, Nir Ben-Zrihem, Shie Mannor
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:1899-1908, 2016.

Abstract

In recent years there is a growing interest in using deep representations for reinforcement learning. In this paper, we present a methodology and tools to analyze Deep Q-networks (DQNs) in a non-blind matter. Using our tools we reveal that the features learned by DQNs aggregate the state space in a hierarchical fashion, explaining its success. Moreover we are able to understand and describe the policies learned by DQNs for three different Atari2600 games and suggest ways to interpret, debug and optimize of deep neural networks in Reinforcement Learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v48-zahavy16, title = {Graying the black box: Understanding DQNs}, author = {Zahavy, Tom and Ben-Zrihem, Nir and Mannor, Shie}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {1899--1908}, year = {2016}, editor = {Balcan, Maria Florina and Weinberger, Kilian Q.}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/zahavy16.pdf}, url = {https://proceedings.mlr.press/v48/zahavy16.html}, abstract = {In recent years there is a growing interest in using deep representations for reinforcement learning. In this paper, we present a methodology and tools to analyze Deep Q-networks (DQNs) in a non-blind matter. Using our tools we reveal that the features learned by DQNs aggregate the state space in a hierarchical fashion, explaining its success. Moreover we are able to understand and describe the policies learned by DQNs for three different Atari2600 games and suggest ways to interpret, debug and optimize of deep neural networks in Reinforcement Learning.} }
Endnote
%0 Conference Paper %T Graying the black box: Understanding DQNs %A Tom Zahavy %A Nir Ben-Zrihem %A Shie Mannor %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-zahavy16 %I PMLR %P 1899--1908 %U https://proceedings.mlr.press/v48/zahavy16.html %V 48 %X In recent years there is a growing interest in using deep representations for reinforcement learning. In this paper, we present a methodology and tools to analyze Deep Q-networks (DQNs) in a non-blind matter. Using our tools we reveal that the features learned by DQNs aggregate the state space in a hierarchical fashion, explaining its success. Moreover we are able to understand and describe the policies learned by DQNs for three different Atari2600 games and suggest ways to interpret, debug and optimize of deep neural networks in Reinforcement Learning.
RIS
TY - CPAPER TI - Graying the black box: Understanding DQNs AU - Tom Zahavy AU - Nir Ben-Zrihem AU - Shie Mannor BT - Proceedings of The 33rd International Conference on Machine Learning DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-zahavy16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 48 SP - 1899 EP - 1908 L1 - http://proceedings.mlr.press/v48/zahavy16.pdf UR - https://proceedings.mlr.press/v48/zahavy16.html AB - In recent years there is a growing interest in using deep representations for reinforcement learning. In this paper, we present a methodology and tools to analyze Deep Q-networks (DQNs) in a non-blind matter. Using our tools we reveal that the features learned by DQNs aggregate the state space in a hierarchical fashion, explaining its success. Moreover we are able to understand and describe the policies learned by DQNs for three different Atari2600 games and suggest ways to interpret, debug and optimize of deep neural networks in Reinforcement Learning. ER -
APA
Zahavy, T., Ben-Zrihem, N. & Mannor, S.. (2016). Graying the black box: Understanding DQNs. Proceedings of The 33rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 48:1899-1908 Available from https://proceedings.mlr.press/v48/zahavy16.html.

Related Material