Defects of Convolutional Decoder Networks in Frequency Representation

Ling Tang, Wen Shen, Zhanpeng Zhou, Yuefeng Chen, Quanshi Zhang
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:33758-33791, 2023.

Abstract

In this paper, we prove the representation defects of a cascaded convolutional decoder network, considering the capacity of representing different frequency components of an input sample. We conduct the discrete Fourier transform on each channel of the feature map in an intermediate layer of the decoder network. Then, we extend the 2D circular convolution theorem to represent the forward and backward propagations through convolutional layers in the frequency domain. Based on this, we prove three defects in representing feature spectrums. First, we prove that the convolution operation, the zero-padding operation, and a set of other settings all make a convolutional decoder network more likely to weaken high-frequency components. Second, we prove that the upsampling operation generates a feature spectrum, in which strong signals repetitively appear at certain frequencies. Third, we prove that if the frequency components in the input sample and frequency components in the target output for regression have a small shift, then the decoder usually cannot be effectively learned.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-tang23i, title = {Defects of Convolutional Decoder Networks in Frequency Representation}, author = {Tang, Ling and Shen, Wen and Zhou, Zhanpeng and Chen, Yuefeng and Zhang, Quanshi}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {33758--33791}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/tang23i/tang23i.pdf}, url = {https://proceedings.mlr.press/v202/tang23i.html}, abstract = {In this paper, we prove the representation defects of a cascaded convolutional decoder network, considering the capacity of representing different frequency components of an input sample. We conduct the discrete Fourier transform on each channel of the feature map in an intermediate layer of the decoder network. Then, we extend the 2D circular convolution theorem to represent the forward and backward propagations through convolutional layers in the frequency domain. Based on this, we prove three defects in representing feature spectrums. First, we prove that the convolution operation, the zero-padding operation, and a set of other settings all make a convolutional decoder network more likely to weaken high-frequency components. Second, we prove that the upsampling operation generates a feature spectrum, in which strong signals repetitively appear at certain frequencies. Third, we prove that if the frequency components in the input sample and frequency components in the target output for regression have a small shift, then the decoder usually cannot be effectively learned.} }
Endnote
%0 Conference Paper %T Defects of Convolutional Decoder Networks in Frequency Representation %A Ling Tang %A Wen Shen %A Zhanpeng Zhou %A Yuefeng Chen %A Quanshi Zhang %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-tang23i %I PMLR %P 33758--33791 %U https://proceedings.mlr.press/v202/tang23i.html %V 202 %X In this paper, we prove the representation defects of a cascaded convolutional decoder network, considering the capacity of representing different frequency components of an input sample. We conduct the discrete Fourier transform on each channel of the feature map in an intermediate layer of the decoder network. Then, we extend the 2D circular convolution theorem to represent the forward and backward propagations through convolutional layers in the frequency domain. Based on this, we prove three defects in representing feature spectrums. First, we prove that the convolution operation, the zero-padding operation, and a set of other settings all make a convolutional decoder network more likely to weaken high-frequency components. Second, we prove that the upsampling operation generates a feature spectrum, in which strong signals repetitively appear at certain frequencies. Third, we prove that if the frequency components in the input sample and frequency components in the target output for regression have a small shift, then the decoder usually cannot be effectively learned.
APA
Tang, L., Shen, W., Zhou, Z., Chen, Y. & Zhang, Q.. (2023). Defects of Convolutional Decoder Networks in Frequency Representation. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:33758-33791 Available from https://proceedings.mlr.press/v202/tang23i.html.

Related Material