Deciding How to Decide: Dynamic Routing in Artificial Neural Networks

Mason McGill, Pietro Perona
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:2363-2372, 2017.

Abstract

We propose and systematically evaluate three strategies for training dynamically-routed artificial neural networks: graphs of learned transformations through which different input signals may take different paths. Though some approaches have advantages over others, the resulting networks are often qualitatively similar. We find that, in dynamically-routed networks trained to classify images, layers and branches become specialized to process distinct categories of images. Additionally, given a fixed computational budget, dynamically-routed networks tend to perform better than comparable statically-routed networks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-mcgill17a, title = {Deciding How to Decide: Dynamic Routing in Artificial Neural Networks}, author = {Mason McGill and Pietro Perona}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {2363--2372}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/mcgill17a/mcgill17a.pdf}, url = {https://proceedings.mlr.press/v70/mcgill17a.html}, abstract = {We propose and systematically evaluate three strategies for training dynamically-routed artificial neural networks: graphs of learned transformations through which different input signals may take different paths. Though some approaches have advantages over others, the resulting networks are often qualitatively similar. We find that, in dynamically-routed networks trained to classify images, layers and branches become specialized to process distinct categories of images. Additionally, given a fixed computational budget, dynamically-routed networks tend to perform better than comparable statically-routed networks.} }
Endnote
%0 Conference Paper %T Deciding How to Decide: Dynamic Routing in Artificial Neural Networks %A Mason McGill %A Pietro Perona %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-mcgill17a %I PMLR %P 2363--2372 %U https://proceedings.mlr.press/v70/mcgill17a.html %V 70 %X We propose and systematically evaluate three strategies for training dynamically-routed artificial neural networks: graphs of learned transformations through which different input signals may take different paths. Though some approaches have advantages over others, the resulting networks are often qualitatively similar. We find that, in dynamically-routed networks trained to classify images, layers and branches become specialized to process distinct categories of images. Additionally, given a fixed computational budget, dynamically-routed networks tend to perform better than comparable statically-routed networks.
APA
McGill, M. & Perona, P.. (2017). Deciding How to Decide: Dynamic Routing in Artificial Neural Networks. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:2363-2372 Available from https://proceedings.mlr.press/v70/mcgill17a.html.

Related Material