IMEXnet A Forward Stable Deep Neural Network

Eldad Haber, Keegan Lensink, Eran Treister, Lars Ruthotto
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:2525-2534, 2019.

Abstract

Deep convolutional neural networks have revolutionized many machine learning and computer vision tasks, however, some remaining key challenges limit their wider use. These challenges include improving the network’s robustness to perturbations of the input image and the limited “field of view” of convolution operators. We introduce the IMEXnet that addresses these challenges by adapting semi-implicit methods for partial differential equations. Compared to similar explicit networks, such as residual networks, our network is more stable, which has recently shown to reduce the sensitivity to small changes in the input features and improve generalization. The addition of an implicit step connects all pixels in each channel of the image and therefore addresses the field of view problem while still being comparable to standard convolutions in terms of the number of parameters and computational complexity. We also present a new dataset for semantic segmentation and demonstrate the effectiveness of our architecture using the NYU Depth dataset.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-haber19a, title = {{IMEX}net A Forward Stable Deep Neural Network}, author = {Haber, Eldad and Lensink, Keegan and Treister, Eran and Ruthotto, Lars}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {2525--2534}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/haber19a/haber19a.pdf}, url = {https://proceedings.mlr.press/v97/haber19a.html}, abstract = {Deep convolutional neural networks have revolutionized many machine learning and computer vision tasks, however, some remaining key challenges limit their wider use. These challenges include improving the network’s robustness to perturbations of the input image and the limited “field of view” of convolution operators. We introduce the IMEXnet that addresses these challenges by adapting semi-implicit methods for partial differential equations. Compared to similar explicit networks, such as residual networks, our network is more stable, which has recently shown to reduce the sensitivity to small changes in the input features and improve generalization. The addition of an implicit step connects all pixels in each channel of the image and therefore addresses the field of view problem while still being comparable to standard convolutions in terms of the number of parameters and computational complexity. We also present a new dataset for semantic segmentation and demonstrate the effectiveness of our architecture using the NYU Depth dataset.} }
Endnote
%0 Conference Paper %T IMEXnet A Forward Stable Deep Neural Network %A Eldad Haber %A Keegan Lensink %A Eran Treister %A Lars Ruthotto %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-haber19a %I PMLR %P 2525--2534 %U https://proceedings.mlr.press/v97/haber19a.html %V 97 %X Deep convolutional neural networks have revolutionized many machine learning and computer vision tasks, however, some remaining key challenges limit their wider use. These challenges include improving the network’s robustness to perturbations of the input image and the limited “field of view” of convolution operators. We introduce the IMEXnet that addresses these challenges by adapting semi-implicit methods for partial differential equations. Compared to similar explicit networks, such as residual networks, our network is more stable, which has recently shown to reduce the sensitivity to small changes in the input features and improve generalization. The addition of an implicit step connects all pixels in each channel of the image and therefore addresses the field of view problem while still being comparable to standard convolutions in terms of the number of parameters and computational complexity. We also present a new dataset for semantic segmentation and demonstrate the effectiveness of our architecture using the NYU Depth dataset.
APA
Haber, E., Lensink, K., Treister, E. & Ruthotto, L.. (2019). IMEXnet A Forward Stable Deep Neural Network. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:2525-2534 Available from https://proceedings.mlr.press/v97/haber19a.html.

Related Material