Gradually Updated Neural Networks for Large-Scale Image Recognition

Siyuan Qiao, Zhishuai Zhang, Wei Shen, Bo Wang, Alan Yuille
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:4188-4197, 2018.

Abstract

Depth is one of the keys that make neural networks succeed in the task of large-scale image recognition. The state-of-the-art network architectures usually increase the depths by cascading convolutional layers or building blocks. In this paper, we present an alternative method to increase the depth. Our method is by introducing computation orderings to the channels within convolutional layers or blocks, based on which we gradually compute the outputs in a channel-wise manner. The added orderings not only increase the depths and the learning capacities of the networks without any additional computation costs, but also eliminate the overlap singularities so that the networks are able to converge faster and perform better. Experiments show that the networks based on our method achieve the state-of-the-art performances on CIFAR and ImageNet datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-qiao18b, title = {Gradually Updated Neural Networks for Large-Scale Image Recognition}, author = {Qiao, Siyuan and Zhang, Zhishuai and Shen, Wei and Wang, Bo and Yuille, Alan}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {4188--4197}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/qiao18b/qiao18b.pdf}, url = {https://proceedings.mlr.press/v80/qiao18b.html}, abstract = {Depth is one of the keys that make neural networks succeed in the task of large-scale image recognition. The state-of-the-art network architectures usually increase the depths by cascading convolutional layers or building blocks. In this paper, we present an alternative method to increase the depth. Our method is by introducing computation orderings to the channels within convolutional layers or blocks, based on which we gradually compute the outputs in a channel-wise manner. The added orderings not only increase the depths and the learning capacities of the networks without any additional computation costs, but also eliminate the overlap singularities so that the networks are able to converge faster and perform better. Experiments show that the networks based on our method achieve the state-of-the-art performances on CIFAR and ImageNet datasets.} }
Endnote
%0 Conference Paper %T Gradually Updated Neural Networks for Large-Scale Image Recognition %A Siyuan Qiao %A Zhishuai Zhang %A Wei Shen %A Bo Wang %A Alan Yuille %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-qiao18b %I PMLR %P 4188--4197 %U https://proceedings.mlr.press/v80/qiao18b.html %V 80 %X Depth is one of the keys that make neural networks succeed in the task of large-scale image recognition. The state-of-the-art network architectures usually increase the depths by cascading convolutional layers or building blocks. In this paper, we present an alternative method to increase the depth. Our method is by introducing computation orderings to the channels within convolutional layers or blocks, based on which we gradually compute the outputs in a channel-wise manner. The added orderings not only increase the depths and the learning capacities of the networks without any additional computation costs, but also eliminate the overlap singularities so that the networks are able to converge faster and perform better. Experiments show that the networks based on our method achieve the state-of-the-art performances on CIFAR and ImageNet datasets.
APA
Qiao, S., Zhang, Z., Shen, W., Wang, B. & Yuille, A.. (2018). Gradually Updated Neural Networks for Large-Scale Image Recognition. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:4188-4197 Available from https://proceedings.mlr.press/v80/qiao18b.html.

Related Material