Randomly Weighted Neuromodulation in Neural Networks Facilitates Learning of Manifolds Common Across Tasks

Jinyung Hong, Theodore P. Pavlic
Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models, PMLR 243:315-325, 2024.

Abstract

Geometric Sensitive Hashing functions, a family of Local Sensitive Hashing functions, are neural network models that learn class-specific manifold geometry in supervised learning. However, given a set of supervised learning tasks, understanding the manifold geometries that can represent each task and the kinds of relationships between the tasks based on them has received little attention. We explore a formalization of this question by considering a generative process where each task is associated with a high-dimensional manifold, which can be done in brain-like models with neuromodulatory systems. Following this formulation, we define Task-specific Geometric Sensitive Hashing and show that a randomly weighted neural network with a neuromodulation system can realize this function.

Cite this Paper

BibTeX
@InProceedings{pmlr-v243-hong24a, title = {Randomly Weighted Neuromodulation in Neural Networks Facilitates Learning of Manifolds Common Across Tasks}, author = {Hong, Jinyung and Pavlic, Theodore P.}, booktitle = {Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models}, pages = {315--325}, year = {2024}, editor = {Fumero, Marco and Rodolá, Emanuele and Domine, Clementine and Locatello, Francesco and Dziugaite, Karolina and Mathilde, Caron}, volume = {243}, series = {Proceedings of Machine Learning Research}, month = {15 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v243/hong24a/hong24a.pdf}, url = {https://proceedings.mlr.press/v243/hong24a.html}, abstract = {Geometric Sensitive Hashing functions, a family of Local Sensitive Hashing functions, are neural network models that learn class-specific manifold geometry in supervised learning. However, given a set of supervised learning tasks, understanding the manifold geometries that can represent each task and the kinds of relationships between the tasks based on them has received little attention. We explore a formalization of this question by considering a generative process where each task is associated with a high-dimensional manifold, which can be done in brain-like models with neuromodulatory systems. Following this formulation, we define Task-specific Geometric Sensitive Hashing and show that a randomly weighted neural network with a neuromodulation system can realize this function.} }
Endnote
%0 Conference Paper %T Randomly Weighted Neuromodulation in Neural Networks Facilitates Learning of Manifolds Common Across Tasks %A Jinyung Hong %A Theodore P. Pavlic %B Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models %C Proceedings of Machine Learning Research %D 2024 %E Marco Fumero %E Emanuele Rodolá %E Clementine Domine %E Francesco Locatello %E Karolina Dziugaite %E Caron Mathilde %F pmlr-v243-hong24a %I PMLR %P 315--325 %U https://proceedings.mlr.press/v243/hong24a.html %V 243 %X Geometric Sensitive Hashing functions, a family of Local Sensitive Hashing functions, are neural network models that learn class-specific manifold geometry in supervised learning. However, given a set of supervised learning tasks, understanding the manifold geometries that can represent each task and the kinds of relationships between the tasks based on them has received little attention. We explore a formalization of this question by considering a generative process where each task is associated with a high-dimensional manifold, which can be done in brain-like models with neuromodulatory systems. Following this formulation, we define Task-specific Geometric Sensitive Hashing and show that a randomly weighted neural network with a neuromodulation system can realize this function.
APA
Hong, J. & Pavlic, T.P.. (2024). Randomly Weighted Neuromodulation in Neural Networks Facilitates Learning of Manifolds Common Across Tasks. Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models, in Proceedings of Machine Learning Research 243:315-325 Available from https://proceedings.mlr.press/v243/hong24a.html.

Related Material