DCFNet: Deep Neural Network with Decomposed Convolutional Filters

Qiang Qiu, Xiuyuan Cheng,  Calderbank, Guillermo Sapiro
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:4198-4207, 2018.

Abstract

Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-qiu18a, title = {{DCFN}et: Deep Neural Network with Decomposed Convolutional Filters}, author = {Qiu, Qiang and Cheng, Xiuyuan and robert Calderbank and Sapiro, Guillermo}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {4198--4207}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/qiu18a/qiu18a.pdf}, url = {https://proceedings.mlr.press/v80/qiu18a.html}, abstract = {Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.} }
Endnote
%0 Conference Paper %T DCFNet: Deep Neural Network with Decomposed Convolutional Filters %A Qiang Qiu %A Xiuyuan Cheng %A Calderbank %A Guillermo Sapiro %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-qiu18a %I PMLR %P 4198--4207 %U https://proceedings.mlr.press/v80/qiu18a.html %V 80 %X Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.
APA
Qiu, Q., Cheng, X., Calderbank, & Sapiro, G.. (2018). DCFNet: Deep Neural Network with Decomposed Convolutional Filters. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:4198-4207 Available from https://proceedings.mlr.press/v80/qiu18a.html.

Related Material