Butterfly-Net2: Simplified Butterfly-Net and Fourier Transform Initialization

Zhongshu Xu, Yingzhou Li, Xiuyuan Cheng
Proceedings of The First Mathematical and Scientific Machine Learning Conference, PMLR 107:431-450, 2020.

Abstract

Structured CNN designed using the prior information of problems potentially improves efficiency over conventional CNNs in various tasks in solving PDEs and inverse problems in signal processing. This paper introduces BNet2, a simplified Butterfly-Net and inline with the conventional CNN. Moreover, a Fourier transform initialization is proposed for both BNet2 and CNN with guaranteed approximation power to represent the Fourier transform operator. Experimentally, BNet2 and the Fourier transform initialization strategy are tested on various tasks, including approximating Fourier transform operator, end-to-end solvers of linear and nonlinear PDEs, and denoising and deblurring of 1D signals. On all tasks, under the same initialization, BNet2 achieves similar accuracy as CNN but has fewer parameters. And Fourier transform initialized BNet2 and CNN consistently improve the training and testing accuracy over the randomly initialized CNN.

Cite this Paper

BibTeX
@InProceedings{pmlr-v107-xu20b, title = {{Butterfly-Net2: Simplified Butterfly-Net and F}ourier Transform Initialization}, author = {Xu, Zhongshu and Li, Yingzhou and Cheng, Xiuyuan}, booktitle = {Proceedings of The First Mathematical and Scientific Machine Learning Conference}, pages = {431--450}, year = {2020}, editor = {Lu, Jianfeng and Ward, Rachel}, volume = {107}, series = {Proceedings of Machine Learning Research}, month = {20--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v107/xu20b/xu20b.pdf}, url = {https://proceedings.mlr.press/v107/xu20b.html}, abstract = {Structured CNN designed using the prior information of problems potentially improves efficiency over conventional CNNs in various tasks in solving PDEs and inverse problems in signal processing. This paper introduces BNet2, a simplified Butterfly-Net and inline with the conventional CNN. Moreover, a Fourier transform initialization is proposed for both BNet2 and CNN with guaranteed approximation power to represent the Fourier transform operator. Experimentally, BNet2 and the Fourier transform initialization strategy are tested on various tasks, including approximating Fourier transform operator, end-to-end solvers of linear and nonlinear PDEs, and denoising and deblurring of 1D signals. On all tasks, under the same initialization, BNet2 achieves similar accuracy as CNN but has fewer parameters. And Fourier transform initialized BNet2 and CNN consistently improve the training and testing accuracy over the randomly initialized CNN. } }
Endnote
%0 Conference Paper %T Butterfly-Net2: Simplified Butterfly-Net and Fourier Transform Initialization %A Zhongshu Xu %A Yingzhou Li %A Xiuyuan Cheng %B Proceedings of The First Mathematical and Scientific Machine Learning Conference %C Proceedings of Machine Learning Research %D 2020 %E Jianfeng Lu %E Rachel Ward %F pmlr-v107-xu20b %I PMLR %P 431--450 %U https://proceedings.mlr.press/v107/xu20b.html %V 107 %X Structured CNN designed using the prior information of problems potentially improves efficiency over conventional CNNs in various tasks in solving PDEs and inverse problems in signal processing. This paper introduces BNet2, a simplified Butterfly-Net and inline with the conventional CNN. Moreover, a Fourier transform initialization is proposed for both BNet2 and CNN with guaranteed approximation power to represent the Fourier transform operator. Experimentally, BNet2 and the Fourier transform initialization strategy are tested on various tasks, including approximating Fourier transform operator, end-to-end solvers of linear and nonlinear PDEs, and denoising and deblurring of 1D signals. On all tasks, under the same initialization, BNet2 achieves similar accuracy as CNN but has fewer parameters. And Fourier transform initialized BNet2 and CNN consistently improve the training and testing accuracy over the randomly initialized CNN.
APA
Xu, Z., Li, Y. & Cheng, X.. (2020). Butterfly-Net2: Simplified Butterfly-Net and Fourier Transform Initialization. Proceedings of The First Mathematical and Scientific Machine Learning Conference, in Proceedings of Machine Learning Research 107:431-450 Available from https://proceedings.mlr.press/v107/xu20b.html.

Related Material