Discrete Deep Feature Extraction: A Theory and New Architectures

Thomas Wiatowski, Michael Tschannen, Aleksandar Stanic, Philipp Grohs, Helmut Boelcskei
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:2149-2158, 2016.

Abstract

First steps towards a mathematical theory of deep convolutional neural networks for feature extraction were made—for the continuous-time case—in Mallat, 2012, and Wiatowski and Bölcskei, 2015. This paper considers the discrete case, introduces new convolutional neural network architectures, and proposes a mathematical framework for their analysis. Specifically, we establish deformation and translation sensitivity results of local and global nature, and we investigate how certain structural properties of the input signal are reflected in the corresponding feature vectors. Our theory applies to general filters and general Lipschitz-continuous non-linearities and pooling operators. Experiments on handwritten digit classification and facial landmark detection—including feature importance evaluation—complement the theoretical findings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v48-wiatowski16, title = {Discrete Deep Feature Extraction: A Theory and New Architectures}, author = {Wiatowski, Thomas and Tschannen, Michael and Stanic, Aleksandar and Grohs, Philipp and Boelcskei, Helmut}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {2149--2158}, year = {2016}, editor = {Balcan, Maria Florina and Weinberger, Kilian Q.}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/wiatowski16.pdf}, url = {https://proceedings.mlr.press/v48/wiatowski16.html}, abstract = {First steps towards a mathematical theory of deep convolutional neural networks for feature extraction were made—for the continuous-time case—in Mallat, 2012, and Wiatowski and Bölcskei, 2015. This paper considers the discrete case, introduces new convolutional neural network architectures, and proposes a mathematical framework for their analysis. Specifically, we establish deformation and translation sensitivity results of local and global nature, and we investigate how certain structural properties of the input signal are reflected in the corresponding feature vectors. Our theory applies to general filters and general Lipschitz-continuous non-linearities and pooling operators. Experiments on handwritten digit classification and facial landmark detection—including feature importance evaluation—complement the theoretical findings.} }
Endnote
%0 Conference Paper %T Discrete Deep Feature Extraction: A Theory and New Architectures %A Thomas Wiatowski %A Michael Tschannen %A Aleksandar Stanic %A Philipp Grohs %A Helmut Boelcskei %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-wiatowski16 %I PMLR %P 2149--2158 %U https://proceedings.mlr.press/v48/wiatowski16.html %V 48 %X First steps towards a mathematical theory of deep convolutional neural networks for feature extraction were made—for the continuous-time case—in Mallat, 2012, and Wiatowski and Bölcskei, 2015. This paper considers the discrete case, introduces new convolutional neural network architectures, and proposes a mathematical framework for their analysis. Specifically, we establish deformation and translation sensitivity results of local and global nature, and we investigate how certain structural properties of the input signal are reflected in the corresponding feature vectors. Our theory applies to general filters and general Lipschitz-continuous non-linearities and pooling operators. Experiments on handwritten digit classification and facial landmark detection—including feature importance evaluation—complement the theoretical findings.
RIS
TY - CPAPER TI - Discrete Deep Feature Extraction: A Theory and New Architectures AU - Thomas Wiatowski AU - Michael Tschannen AU - Aleksandar Stanic AU - Philipp Grohs AU - Helmut Boelcskei BT - Proceedings of The 33rd International Conference on Machine Learning DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-wiatowski16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 48 SP - 2149 EP - 2158 L1 - http://proceedings.mlr.press/v48/wiatowski16.pdf UR - https://proceedings.mlr.press/v48/wiatowski16.html AB - First steps towards a mathematical theory of deep convolutional neural networks for feature extraction were made—for the continuous-time case—in Mallat, 2012, and Wiatowski and Bölcskei, 2015. This paper considers the discrete case, introduces new convolutional neural network architectures, and proposes a mathematical framework for their analysis. Specifically, we establish deformation and translation sensitivity results of local and global nature, and we investigate how certain structural properties of the input signal are reflected in the corresponding feature vectors. Our theory applies to general filters and general Lipschitz-continuous non-linearities and pooling operators. Experiments on handwritten digit classification and facial landmark detection—including feature importance evaluation—complement the theoretical findings. ER -
APA
Wiatowski, T., Tschannen, M., Stanic, A., Grohs, P. & Boelcskei, H.. (2016). Discrete Deep Feature Extraction: A Theory and New Architectures. Proceedings of The 33rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 48:2149-2158 Available from https://proceedings.mlr.press/v48/wiatowski16.html.

Related Material