A Natural Lottery Ticket Winner: Reinforcement Learning with Ordinary Neural Circuits

Ramin Hasani, Mathias Lechner, Alexander Amini, Daniela Rus, Radu Grosu
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:4082-4093, 2020.

Abstract

We propose a neural information processing system obtained by re-purposing the function of a biological neural circuit model to govern simulated and real-world control tasks. Inspired by the structure of the nervous system of the soil-worm, C. elegans, we introduce ordinary neural circuits (ONCs), defined as the model of biological neural circuits reparameterized for the control of alternative tasks. We first demonstrate that ONCs realize networks with higher maximum flow compared to arbitrary wired networks. We then learn instances of ONCs to control a series of robotic tasks, including the autonomous parking of a real-world rover robot. For reconfiguration of the purpose of the neural circuit, we adopt a search-based optimization algorithm. Ordinary neural circuits perform on par and, in some cases, significantly surpass the performance of contemporary deep learning models. ONC networks are compact, 77% sparser than their counterpart neural controllers, and their neural dynamics are fully interpretable at the cell-level.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-hasani20a, title = {A Natural Lottery Ticket Winner: Reinforcement Learning with Ordinary Neural Circuits}, author = {Hasani, Ramin and Lechner, Mathias and Amini, Alexander and Rus, Daniela and Grosu, Radu}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {4082--4093}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/hasani20a/hasani20a.pdf}, url = {https://proceedings.mlr.press/v119/hasani20a.html}, abstract = {We propose a neural information processing system obtained by re-purposing the function of a biological neural circuit model to govern simulated and real-world control tasks. Inspired by the structure of the nervous system of the soil-worm, C. elegans, we introduce ordinary neural circuits (ONCs), defined as the model of biological neural circuits reparameterized for the control of alternative tasks. We first demonstrate that ONCs realize networks with higher maximum flow compared to arbitrary wired networks. We then learn instances of ONCs to control a series of robotic tasks, including the autonomous parking of a real-world rover robot. For reconfiguration of the purpose of the neural circuit, we adopt a search-based optimization algorithm. Ordinary neural circuits perform on par and, in some cases, significantly surpass the performance of contemporary deep learning models. ONC networks are compact, 77% sparser than their counterpart neural controllers, and their neural dynamics are fully interpretable at the cell-level.} }
Endnote
%0 Conference Paper %T A Natural Lottery Ticket Winner: Reinforcement Learning with Ordinary Neural Circuits %A Ramin Hasani %A Mathias Lechner %A Alexander Amini %A Daniela Rus %A Radu Grosu %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-hasani20a %I PMLR %P 4082--4093 %U https://proceedings.mlr.press/v119/hasani20a.html %V 119 %X We propose a neural information processing system obtained by re-purposing the function of a biological neural circuit model to govern simulated and real-world control tasks. Inspired by the structure of the nervous system of the soil-worm, C. elegans, we introduce ordinary neural circuits (ONCs), defined as the model of biological neural circuits reparameterized for the control of alternative tasks. We first demonstrate that ONCs realize networks with higher maximum flow compared to arbitrary wired networks. We then learn instances of ONCs to control a series of robotic tasks, including the autonomous parking of a real-world rover robot. For reconfiguration of the purpose of the neural circuit, we adopt a search-based optimization algorithm. Ordinary neural circuits perform on par and, in some cases, significantly surpass the performance of contemporary deep learning models. ONC networks are compact, 77% sparser than their counterpart neural controllers, and their neural dynamics are fully interpretable at the cell-level.
APA
Hasani, R., Lechner, M., Amini, A., Rus, D. & Grosu, R.. (2020). A Natural Lottery Ticket Winner: Reinforcement Learning with Ordinary Neural Circuits. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:4082-4093 Available from https://proceedings.mlr.press/v119/hasani20a.html.

Related Material