Feature Reinforcement Learning using Looping Suffix Trees


Mayank Daswani, Peter Sunehag, Marcus Hutter ;
Proceedings of the Tenth European Workshop on Reinforcement Learning, PMLR 24:11-24, 2013.


There has recently been much interest in history-based methods using suffix trees to solve POMDPs. However, these suffix trees cannot efficiently represent environments that have long-term dependencies. We extend the recently introduced CTΦMDP algorithm to the space of looping suffix trees which have previously only been used in solving deterministic POMDPs. The resulting algorithm replicates results from CTΦMDP for environments with short term dependencies, while it outperforms LSTM-based methods on TMaze, a deep memory environment.

Related Material