Regret Minimization for Partially Observable Deep Reinforcement Learning
[edit]
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:23422351, 2018.
Abstract
Deep reinforcement learning algorithms that estimate state and stateaction value functions have been shown to be effective in a variety of challenging domains, including learning control strategies from raw image pixels. However, algorithms that estimate state and stateaction value functions typically assume a fully observed state and must compensate for partial observations by using finite length observation histories or recurrent networks. In this work, we propose a new deep reinforcement learning algorithm based on counterfactual regret minimization that iteratively updates an approximation to an advantagelike function and is robust to partially observed state. We demonstrate that this new algorithm can substantially outperform strong baseline methods on several partially observed reinforcement learning tasks: learning firstperson 3D navigation in Doom and Minecraft, and acting in the presence of partially observed objects in Doom and Pong.
Related Material


