Playing Atari with Hybrid Quantum-Classical Reinforcement Learning

Owen Lockwood, Mei Si
NeurIPS 2020 Workshop on Pre-registration in Machine Learning, PMLR 148:285-301, 2021.

Abstract

Despite the successes of recent works in quantum reinforcement learning, there are still severe limitations on its applications due to the challenge of encoding large observation spaces into quantum systems. To address this challenge, we propose using a neural network as a data encoder, with the Atari games as our testbed. Specifically, the neural network converts the pixel input from the games to quantum data for a Quantum Variational Circuit (QVC); this hybrid model is then used as a function approximator in the Double Deep Q Networks algorithm. We explore a number of variations of this algorithm and find that our proposed hybrid models do not achieve meaningful results on two Atari games – Breakout and Pong. We suspect this is due to the significantly reduced sizes of the hybrid quantum-classical systems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v148-lockwood21a, title = {Playing Atari with Hybrid Quantum-Classical Reinforcement Learning}, author = {Lockwood, Owen and Si, Mei}, booktitle = {NeurIPS 2020 Workshop on Pre-registration in Machine Learning}, pages = {285--301}, year = {2021}, editor = {Bertinetto, Luca and Henriques, João F. and Albanie, Samuel and Paganini, Michela and Varol, Gül}, volume = {148}, series = {Proceedings of Machine Learning Research}, month = {11 Dec}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v148/lockwood21a/lockwood21a.pdf}, url = {https://proceedings.mlr.press/v148/lockwood21a.html}, abstract = {Despite the successes of recent works in quantum reinforcement learning, there are still severe limitations on its applications due to the challenge of encoding large observation spaces into quantum systems. To address this challenge, we propose using a neural network as a data encoder, with the Atari games as our testbed. Specifically, the neural network converts the pixel input from the games to quantum data for a Quantum Variational Circuit (QVC); this hybrid model is then used as a function approximator in the Double Deep Q Networks algorithm. We explore a number of variations of this algorithm and find that our proposed hybrid models do not achieve meaningful results on two Atari games – Breakout and Pong. We suspect this is due to the significantly reduced sizes of the hybrid quantum-classical systems.} }
Endnote
%0 Conference Paper %T Playing Atari with Hybrid Quantum-Classical Reinforcement Learning %A Owen Lockwood %A Mei Si %B NeurIPS 2020 Workshop on Pre-registration in Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Luca Bertinetto %E João F. Henriques %E Samuel Albanie %E Michela Paganini %E Gül Varol %F pmlr-v148-lockwood21a %I PMLR %P 285--301 %U https://proceedings.mlr.press/v148/lockwood21a.html %V 148 %X Despite the successes of recent works in quantum reinforcement learning, there are still severe limitations on its applications due to the challenge of encoding large observation spaces into quantum systems. To address this challenge, we propose using a neural network as a data encoder, with the Atari games as our testbed. Specifically, the neural network converts the pixel input from the games to quantum data for a Quantum Variational Circuit (QVC); this hybrid model is then used as a function approximator in the Double Deep Q Networks algorithm. We explore a number of variations of this algorithm and find that our proposed hybrid models do not achieve meaningful results on two Atari games – Breakout and Pong. We suspect this is due to the significantly reduced sizes of the hybrid quantum-classical systems.
APA
Lockwood, O. & Si, M.. (2021). Playing Atari with Hybrid Quantum-Classical Reinforcement Learning. NeurIPS 2020 Workshop on Pre-registration in Machine Learning, in Proceedings of Machine Learning Research 148:285-301 Available from https://proceedings.mlr.press/v148/lockwood21a.html.

Related Material