A Modern Self-Referential Weight Matrix That Learns to Modify Itself

Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:9660-9677, 2022.

Abstract

The weight matrix (WM) of a neural network (NN) is its program. The programs of many traditional NNs are learned through gradient descent in some error function, then remain fixed. The WM of a self-referential NN, however, can keep rapidly modifying all of itself during runtime. In principle, such NNs can meta-learn to learn, and meta-meta-learn to meta-learn to learn, and so on, in the sense of recursive self-improvement. While NN architectures potentially capable of implementing such behaviour have been proposed since the ’90s, there have been few if any practical studies. Here we revisit such NNs, building upon recent successes of fast weight programmers and closely related linear Transformers. We propose a scalable self-referential WM (SRWM) that learns to use outer products and the delta update rule to modify itself. We evaluate our SRWM in supervised few-shot learning and in multi-task reinforcement learning with procedurally generated game environments. Our experiments demonstrate both practical applicability and competitive performance of the proposed SRWM. Our code is public.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-irie22b, title = {A Modern Self-Referential Weight Matrix That Learns to Modify Itself}, author = {Irie, Kazuki and Schlag, Imanol and Csord{\'a}s, R{\'o}bert and Schmidhuber, J{\"u}rgen}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {9660--9677}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/irie22b/irie22b.pdf}, url = {https://proceedings.mlr.press/v162/irie22b.html}, abstract = {The weight matrix (WM) of a neural network (NN) is its program. The programs of many traditional NNs are learned through gradient descent in some error function, then remain fixed. The WM of a self-referential NN, however, can keep rapidly modifying all of itself during runtime. In principle, such NNs can meta-learn to learn, and meta-meta-learn to meta-learn to learn, and so on, in the sense of recursive self-improvement. While NN architectures potentially capable of implementing such behaviour have been proposed since the ’90s, there have been few if any practical studies. Here we revisit such NNs, building upon recent successes of fast weight programmers and closely related linear Transformers. We propose a scalable self-referential WM (SRWM) that learns to use outer products and the delta update rule to modify itself. We evaluate our SRWM in supervised few-shot learning and in multi-task reinforcement learning with procedurally generated game environments. Our experiments demonstrate both practical applicability and competitive performance of the proposed SRWM. Our code is public.} }
Endnote
%0 Conference Paper %T A Modern Self-Referential Weight Matrix That Learns to Modify Itself %A Kazuki Irie %A Imanol Schlag %A Róbert Csordás %A Jürgen Schmidhuber %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-irie22b %I PMLR %P 9660--9677 %U https://proceedings.mlr.press/v162/irie22b.html %V 162 %X The weight matrix (WM) of a neural network (NN) is its program. The programs of many traditional NNs are learned through gradient descent in some error function, then remain fixed. The WM of a self-referential NN, however, can keep rapidly modifying all of itself during runtime. In principle, such NNs can meta-learn to learn, and meta-meta-learn to meta-learn to learn, and so on, in the sense of recursive self-improvement. While NN architectures potentially capable of implementing such behaviour have been proposed since the ’90s, there have been few if any practical studies. Here we revisit such NNs, building upon recent successes of fast weight programmers and closely related linear Transformers. We propose a scalable self-referential WM (SRWM) that learns to use outer products and the delta update rule to modify itself. We evaluate our SRWM in supervised few-shot learning and in multi-task reinforcement learning with procedurally generated game environments. Our experiments demonstrate both practical applicability and competitive performance of the proposed SRWM. Our code is public.
APA
Irie, K., Schlag, I., Csordás, R. & Schmidhuber, J.. (2022). A Modern Self-Referential Weight Matrix That Learns to Modify Itself. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:9660-9677 Available from https://proceedings.mlr.press/v162/irie22b.html.

Related Material