ReSet: Learning Recurrent Dynamic Routing in ResNet-like Neural Networks

Iurii Kemaev, Daniil Polykovskiy, Dmitry Vetrov
Proceedings of The 10th Asian Conference on Machine Learning, PMLR 95:422-437, 2018.

Abstract

Neural Network is a powerful Machine Learning tool that shows outstanding performance in Computer Vision, Natural Language Processing, and Artificial Intelligence. In particular, recently proposed ResNet architecture and its modifications produce state-of-the-art results in image classification problems. ResNet and most of the previously proposed architectures have a fixed structure and apply the same transformation to all input images. In this work, we develop a ResNet-based model that dynamically selects Computational Units (CU) for each input object from a learned set of transformations. Dynamic selection allows the network to learn a sequence of useful transformations and apply only required units to predict the image label. We compare our model to ResNet-38 architecture and achieve better results than the original ResNet on CIFAR-10.1 test set. While examining the produced paths, we discovered that the network learned different routes for images from different classes and similar routes for similar images.

Cite this Paper

BibTeX
@InProceedings{pmlr-v95-kemaev18a, title = {ReSet: Learning Recurrent Dynamic Routing in ResNet-like Neural Networks}, author = {Kemaev, Iurii and Polykovskiy, Daniil and Vetrov, Dmitry}, booktitle = {Proceedings of The 10th Asian Conference on Machine Learning}, pages = {422--437}, year = {2018}, editor = {Zhu, Jun and Takeuchi, Ichiro}, volume = {95}, series = {Proceedings of Machine Learning Research}, month = {14--16 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v95/kemaev18a/kemaev18a.pdf}, url = {https://proceedings.mlr.press/v95/kemaev18a.html}, abstract = {Neural Network is a powerful Machine Learning tool that shows outstanding performance in Computer Vision, Natural Language Processing, and Artificial Intelligence. In particular, recently proposed ResNet architecture and its modifications produce state-of-the-art results in image classification problems. ResNet and most of the previously proposed architectures have a fixed structure and apply the same transformation to all input images. In this work, we develop a ResNet-based model that dynamically selects Computational Units (CU) for each input object from a learned set of transformations. Dynamic selection allows the network to learn a sequence of useful transformations and apply only required units to predict the image label. We compare our model to ResNet-38 architecture and achieve better results than the original ResNet on CIFAR-10.1 test set. While examining the produced paths, we discovered that the network learned different routes for images from different classes and similar routes for similar images.} }
Endnote
%0 Conference Paper %T ReSet: Learning Recurrent Dynamic Routing in ResNet-like Neural Networks %A Iurii Kemaev %A Daniil Polykovskiy %A Dmitry Vetrov %B Proceedings of The 10th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jun Zhu %E Ichiro Takeuchi %F pmlr-v95-kemaev18a %I PMLR %P 422--437 %U https://proceedings.mlr.press/v95/kemaev18a.html %V 95 %X Neural Network is a powerful Machine Learning tool that shows outstanding performance in Computer Vision, Natural Language Processing, and Artificial Intelligence. In particular, recently proposed ResNet architecture and its modifications produce state-of-the-art results in image classification problems. ResNet and most of the previously proposed architectures have a fixed structure and apply the same transformation to all input images. In this work, we develop a ResNet-based model that dynamically selects Computational Units (CU) for each input object from a learned set of transformations. Dynamic selection allows the network to learn a sequence of useful transformations and apply only required units to predict the image label. We compare our model to ResNet-38 architecture and achieve better results than the original ResNet on CIFAR-10.1 test set. While examining the produced paths, we discovered that the network learned different routes for images from different classes and similar routes for similar images.
APA
Kemaev, I., Polykovskiy, D. & Vetrov, D.. (2018). ReSet: Learning Recurrent Dynamic Routing in ResNet-like Neural Networks. Proceedings of The 10th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 95:422-437 Available from https://proceedings.mlr.press/v95/kemaev18a.html.

Related Material