Self-Supervised Learning for Multi-Goal Grid World: Comparing Leela and Deep Q Network

Steve Kommrusch
Proceedings of the First International Workshop on Self-Supervised Learning, PMLR 131:72-88, 2020.

Abstract

Modern machine learning research has explored numerous approaches to solving reinforce- ment learning with multiple goals and sparse rewards as well as learning correct actions from a small number of exploratory samples. We explore the ability of a self-supervised system which automatically creates and tests symbolic hypotheses about the world to ad- dress these same issues. Leela is a system which builds an understanding of the world using constructivist artificial intelligence. For our study, we create an N ∗ N grid world with goals related to proprioceptive or visual positions for exploration. We compare Leela to a DQN which includes hindsight for improving multigoal learning with sparse rewards. Our results show that Leela is able to learn to solve multigoal problems in an N ∗ N world with approximately 160N2 exploratory steps compared to 360N2.7 steps required by the DQN.

Cite this Paper

BibTeX
@InProceedings{pmlr-v131-kommrusch20a, title = {Self-Supervised Learning for Multi-Goal Grid World: Comparing Leela and Deep Q Network}, author = {Kommrusch, Steve}, booktitle = {Proceedings of the First International Workshop on Self-Supervised Learning}, pages = {72--88}, year = {2020}, editor = {Minsky, Henry and Robertson, Paul and Georgeon, Olivier L. and Minsky, Milan and Shaoul, Cyrus}, volume = {131}, series = {Proceedings of Machine Learning Research}, month = {27--28 Feb}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v131/kommrusch20a/kommrusch20a.pdf}, url = {https://proceedings.mlr.press/v131/kommrusch20a.html}, abstract = {Modern machine learning research has explored numerous approaches to solving reinforce- ment learning with multiple goals and sparse rewards as well as learning correct actions from a small number of exploratory samples. We explore the ability of a self-supervised system which automatically creates and tests symbolic hypotheses about the world to ad- dress these same issues. Leela is a system which builds an understanding of the world using constructivist artificial intelligence. For our study, we create an N ∗ N grid world with goals related to proprioceptive or visual positions for exploration. We compare Leela to a DQN which includes hindsight for improving multigoal learning with sparse rewards. Our results show that Leela is able to learn to solve multigoal problems in an N ∗ N world with approximately 160N2 exploratory steps compared to 360N2.7 steps required by the DQN.} }
Endnote
%0 Conference Paper %T Self-Supervised Learning for Multi-Goal Grid World: Comparing Leela and Deep Q Network %A Steve Kommrusch %B Proceedings of the First International Workshop on Self-Supervised Learning %C Proceedings of Machine Learning Research %D 2020 %E Henry Minsky %E Paul Robertson %E Olivier L. Georgeon %E Milan Minsky %E Cyrus Shaoul %F pmlr-v131-kommrusch20a %I PMLR %P 72--88 %U https://proceedings.mlr.press/v131/kommrusch20a.html %V 131 %X Modern machine learning research has explored numerous approaches to solving reinforce- ment learning with multiple goals and sparse rewards as well as learning correct actions from a small number of exploratory samples. We explore the ability of a self-supervised system which automatically creates and tests symbolic hypotheses about the world to ad- dress these same issues. Leela is a system which builds an understanding of the world using constructivist artificial intelligence. For our study, we create an N ∗ N grid world with goals related to proprioceptive or visual positions for exploration. We compare Leela to a DQN which includes hindsight for improving multigoal learning with sparse rewards. Our results show that Leela is able to learn to solve multigoal problems in an N ∗ N world with approximately 160N2 exploratory steps compared to 360N2.7 steps required by the DQN.
APA
Kommrusch, S.. (2020). Self-Supervised Learning for Multi-Goal Grid World: Comparing Leela and Deep Q Network. Proceedings of the First International Workshop on Self-Supervised Learning, in Proceedings of Machine Learning Research 131:72-88 Available from https://proceedings.mlr.press/v131/kommrusch20a.html.

Related Material