Polarity Is All You Need to Learn and Transfer Faster

Qingyang Wang, Michael Alan Powell, Eric W Bridgeford, Ali Geisa, Joshua T Vogelstein
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:36264-36284, 2023.

Abstract

Natural intelligences (NIs) thrive in a dynamic world - they learn quickly, sometimes with only a few samples. In contrast, artificial intelligences (AIs) typically learn with a prohibitive number of training samples and computational power. What design principle difference between NI and AI could contribute to such a discrepancy? Here, we investigate the role of weight polarity: development processes initialize NIs with advantageous polarity configurations; as NIs grow and learn, synapse magnitudes update, yet polarities are largely kept unchanged. We demonstrate with simulation and image classification tasks that if weight polarities are adequately set a priori, then networks learn with less time and data. We also explicitly illustrate situations in which a priori setting the weight polarities is disadvantageous for networks. Our work illustrates the value of weight polarities from the perspective of statistical and computational efficiency during learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-wang23ae, title = {Polarity Is All You Need to Learn and Transfer Faster}, author = {Wang, Qingyang and Powell, Michael Alan and Bridgeford, Eric W and Geisa, Ali and Vogelstein, Joshua T}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {36264--36284}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/wang23ae/wang23ae.pdf}, url = {https://proceedings.mlr.press/v202/wang23ae.html}, abstract = {Natural intelligences (NIs) thrive in a dynamic world - they learn quickly, sometimes with only a few samples. In contrast, artificial intelligences (AIs) typically learn with a prohibitive number of training samples and computational power. What design principle difference between NI and AI could contribute to such a discrepancy? Here, we investigate the role of weight polarity: development processes initialize NIs with advantageous polarity configurations; as NIs grow and learn, synapse magnitudes update, yet polarities are largely kept unchanged. We demonstrate with simulation and image classification tasks that if weight polarities are adequately set a priori, then networks learn with less time and data. We also explicitly illustrate situations in which a priori setting the weight polarities is disadvantageous for networks. Our work illustrates the value of weight polarities from the perspective of statistical and computational efficiency during learning.} }
Endnote
%0 Conference Paper %T Polarity Is All You Need to Learn and Transfer Faster %A Qingyang Wang %A Michael Alan Powell %A Eric W Bridgeford %A Ali Geisa %A Joshua T Vogelstein %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-wang23ae %I PMLR %P 36264--36284 %U https://proceedings.mlr.press/v202/wang23ae.html %V 202 %X Natural intelligences (NIs) thrive in a dynamic world - they learn quickly, sometimes with only a few samples. In contrast, artificial intelligences (AIs) typically learn with a prohibitive number of training samples and computational power. What design principle difference between NI and AI could contribute to such a discrepancy? Here, we investigate the role of weight polarity: development processes initialize NIs with advantageous polarity configurations; as NIs grow and learn, synapse magnitudes update, yet polarities are largely kept unchanged. We demonstrate with simulation and image classification tasks that if weight polarities are adequately set a priori, then networks learn with less time and data. We also explicitly illustrate situations in which a priori setting the weight polarities is disadvantageous for networks. Our work illustrates the value of weight polarities from the perspective of statistical and computational efficiency during learning.
APA
Wang, Q., Powell, M.A., Bridgeford, E.W., Geisa, A. & Vogelstein, J.T.. (2023). Polarity Is All You Need to Learn and Transfer Faster. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:36264-36284 Available from https://proceedings.mlr.press/v202/wang23ae.html.

Related Material