- title: 'IWSSL Introduction to this volume' abstract: 'This collection of papers was presented at the second annual international workshop on self- supervised learning(IWSSL2021 held virtually, between August 13 and August 14, 2021. They represent the state of the art in an expanding field of research that attempts to build systems that can learn without human intervention with little or no hard-wired domain knowledge, as would a new-born child or animal.' volume: 159 URL: https://proceedings.mlr.press/v159/thorisson22a.html PDF: https://proceedings.mlr.press/v159/thorisson22a/thorisson22a.pdf edit: https://github.com/mlresearch//v159/edit/gh-pages/_posts/2022-04-27-thorisson22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Second International Workshop on Self-Supervised Learning' publisher: 'PMLR' author: - given: Kristinn R. family: Thórisson - given: Paul family: Robertson editor: - given: Kristinn R. family: Thórisson - given: Paul family: Robertson page: 1-4 id: thorisson22a issued: date-parts: - 2022 - 4 - 27 firstpage: 1 lastpage: 4 published: 2022-04-27 00:00:00 +0000 - title: 'The "Explanation Hypothesis" in General Self-Supervised Learning' abstract: 'Self-supervised learning is the ability of an agent to improve its own performance, with respect to one or more goals related to one or more phenomena, without outside help from a teacher or other external aid tailored to the agent’s learning progress. A general learner’s learning process is not limited to a strict set of topics, tasks, or domains. Selfsupervised and general learning machines are still in the early stages of development, as are learning machines that can explain their own knowledge, goals, actions, and reasoning. Research on explanation proper has to date been largely limited to the field of philosophy of science. In this paper I present the hypothesis that general self-supervised learning requires (a particular kind of) explanation generation, and review some key arguments for and against it. Named the explanation hypothesis (ExH), the claim rests on three main pillars. First, that any good explanation of a phenomenon requires reference to relations between sub-parts of that phenomenon, as well as to its context (other phenomena and their parts), especially (but not only) causal relations. Second, that self-supervised general learning of a new phenomenon requires (a kind of) bootstrapping, and that this - and subsequent improvement on the initial knowledge thus produced - relies on reasoning processes. Third, that general self-supervised learning relies on reification of prior knowledge and knowledge-generation processes, which can only be implemented through appropriate reflection mechanisms, whereby current knowledge and prior learning progress is available for explicit inspection by the learning system itself, to be analyzed for use in future learning. The claim thus construed has several important implications for the implementation of general machine intelligence, including that it will neither be achieved without reflection (meta-cognition) nor explicit representation of causal relations, and that internal explanation generation must be a fundamental principle of their operation.' volume: 159 URL: https://proceedings.mlr.press/v159/thorisson22b.html PDF: https://proceedings.mlr.press/v159/thorisson22b/thorisson22b.pdf edit: https://github.com/mlresearch//v159/edit/gh-pages/_posts/2022-04-27-thorisson22b.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Second International Workshop on Self-Supervised Learning' publisher: 'PMLR' author: - given: Kristinn R. family: Thórisson editor: - given: Kristinn R. family: Thórisson - given: Paul family: Robertson page: 5-27 id: thorisson22b issued: date-parts: - 2022 - 4 - 27 firstpage: 5 lastpage: 27 published: 2022-04-27 00:00:00 +0000 - title: 'A Unified Model of Reasoning and Learning' abstract: 'We present a novel approach to state space discretization for constructivist and reinforcement learning. Constructivist learning and reinforcement learning often operate on a predefined set of states and transitions (state space). AI researchers design algorithms to reach particular goal states in this state space (for example, visualized in the form of goal cells that a robot should reach in a grid). When the size and the dimensionality of the state space increases, however, finding goal states becomes intractable. It is nonetheless assumed that these algorithms can have useful applications in the physical world provided that there is a way to construct a discrete state space of reasonable size and dimensionality. Yet, the manner in which the state space is discretized is the source of many problems for both constructivist and reinforcement learning approaches. The problems can roughly be divided into two categories: (1) wiring too much domain information into the solution, and (2) requiring massive storage to represent the state space (such as Q-tables. The problems relate to (1) the non generality arising from wiring domain information into the solution, and (2) non scalability of the approach to useful domains involving high dimensional state spaces. Another important limitation is that high dimensional state spaces require a massive number of learning trials. We present a new approach that builds upon ideas from place cells and cognitive maps.' volume: 159 URL: https://proceedings.mlr.press/v159/wang22a.html PDF: https://proceedings.mlr.press/v159/wang22a/wang22a.pdf edit: https://github.com/mlresearch//v159/edit/gh-pages/_posts/2022-04-27-wang22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Second International Workshop on Self-Supervised Learning' publisher: 'PMLR' author: - given: Pei family: Wang editor: - given: Kristinn R. family: Thórisson - given: Paul family: Robertson page: 28-48 id: wang22a issued: date-parts: - 2022 - 4 - 27 firstpage: 28 lastpage: 48 published: 2022-04-27 00:00:00 +0000 - title: 'Comparison of Machine Learners on an ABA Experiment Format of the Cart-Pole Task' abstract: 'Current approaches to online learning focus primarily on reinforcement learning (RL) - algorithms that learn through feedback from experience. While most current RL algorithms have shown good results in learning to perform tasks for which they were specifically designed, most of them lack a level of generalization needed to use existing knowledge to handle novel situations - a property referred to as autonomous transfer learning. Situations encountered by such systems which were not present during the training phase can lead to critical failure. In the present research we analyzed the autonomous transfer learning capabilities of five different machine learning approaches - i.e. an Actor-Critic, a Q-Learner, a Policy Gradient Learner, a Double-Deep Q-Learner, and OpenNARS for Applications. Following a classic ABA experimental format, the learners were all trained on the well-known cart-pole task in phase A-1, before strategic changes to the task were introduced in phase B, consisting of inverting the direction of control of the cart (move-left command moved the cart to the right and vice versa), as well as the introduction of noise. All analyzed learners show an extreme performance drop when the action command is inverted in phase B, resulting in long (re-)training periods trying to reach A1 performance. Most learners do not reach initial A1 performance levels in phase B, some falling very far from them. Furthermore, previously learned knowledge is not retained during the re-training, resulting in an even larger performance drop when the task is changed back to the original settings in phase A2. Only one learner (NARS) reached comparable performance in A1 and A2, demonstrating retention of, and return to, priorly-acquired knowledge.' volume: 159 URL: https://proceedings.mlr.press/v159/eberding22a.html PDF: https://proceedings.mlr.press/v159/eberding22a/eberding22a.pdf edit: https://github.com/mlresearch//v159/edit/gh-pages/_posts/2022-04-27-eberding22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Second International Workshop on Self-Supervised Learning' publisher: 'PMLR' author: - given: Leonard M. family: Eberding editor: - given: Kristinn R. family: Thórisson - given: Paul family: Robertson page: 49-63 id: eberding22a issued: date-parts: - 2022 - 4 - 27 firstpage: 49 lastpage: 63 published: 2022-04-27 00:00:00 +0000 - title: 'Artificial Emotions for Rapid Online Explorative Learning' abstract: 'For decades, A.I. has been able to produce impressive results on hard problems, such as games playing in synthetic environments, but have had difficulty in interfacing with the natural world. Recently machine learning has enabled A.I. to interface more robustly with the real world. Statistical methods for speech understanding opened the door to voice-based systems and more recently deep-learning has revolutionized computer vision to the extent that wild speculation now predicts artificial superintelligence surpassing human intelligence, but we are a few major breakthroughs short of that being achieved. We know what some of these breakthroughs need to be. We need to replace supervised learning with unsupervised learning and we need to take on topics like motivation, attention, and emotions. In this article, we describe an architecture that touches on some of these issues drawing inspiration from neuroscience. We describe three aspects of the architecture in this article that address learning through fear and reward and address the focus of attention. These three systems are intimately linked in mammalian brains. We believe that this work represents an attempt to bridge the gap between high order reasoning and base-level support for motivation and learning in robots.' volume: 159 URL: https://proceedings.mlr.press/v159/robertson22a.html PDF: https://proceedings.mlr.press/v159/robertson22a/robertson22a.pdf edit: https://github.com/mlresearch//v159/edit/gh-pages/_posts/2022-04-27-robertson22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Second International Workshop on Self-Supervised Learning' publisher: 'PMLR' author: - given: Paul family: Robertson editor: - given: Kristinn R. family: Thórisson - given: Paul family: Robertson page: 63-83 id: robertson22a issued: date-parts: - 2022 - 4 - 27 firstpage: 63 lastpage: 83 published: 2022-04-27 00:00:00 +0000