Fourmer: An Efficient Global Modeling Paradigm for Image Restoration

Man Zhou, Jie Huang, Chun-Le Guo, Chongyi Li
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:42589-42601, 2023.

Abstract

Global modeling-based image restoration frameworks have become popular. However, they often require a high memory footprint and do not consider task-specific degradation. Our work presents an alternative approach to global modeling that is more efficient for image restoration. The key insights which motivate our study are two-fold: 1) Fourier transform is capable of disentangling image degradation and content component to a certain extent, serving as the image degradation prior, and 2) Fourier domain innately embraces global properties, where each pixel in the Fourier space is involved with all spatial pixels. While adhering to the “spatial interaction + channel evolution” rule of previous studies, we customize the core designs with Fourier spatial interaction modeling and Fourier channel evolution. Our paradigm, Fourmer, achieves competitive performance on common image restoration tasks such as image de-raining, image enhancement, image dehazing, and guided image super-resolution, while requiring fewer computational resources. The code for Fourmer will be made publicly available.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-zhou23f, title = {Fourmer: An Efficient Global Modeling Paradigm for Image Restoration}, author = {Zhou, Man and Huang, Jie and Guo, Chun-Le and Li, Chongyi}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {42589--42601}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/zhou23f/zhou23f.pdf}, url = {https://proceedings.mlr.press/v202/zhou23f.html}, abstract = {Global modeling-based image restoration frameworks have become popular. However, they often require a high memory footprint and do not consider task-specific degradation. Our work presents an alternative approach to global modeling that is more efficient for image restoration. The key insights which motivate our study are two-fold: 1) Fourier transform is capable of disentangling image degradation and content component to a certain extent, serving as the image degradation prior, and 2) Fourier domain innately embraces global properties, where each pixel in the Fourier space is involved with all spatial pixels. While adhering to the “spatial interaction + channel evolution” rule of previous studies, we customize the core designs with Fourier spatial interaction modeling and Fourier channel evolution. Our paradigm, Fourmer, achieves competitive performance on common image restoration tasks such as image de-raining, image enhancement, image dehazing, and guided image super-resolution, while requiring fewer computational resources. The code for Fourmer will be made publicly available.} }
Endnote
%0 Conference Paper %T Fourmer: An Efficient Global Modeling Paradigm for Image Restoration %A Man Zhou %A Jie Huang %A Chun-Le Guo %A Chongyi Li %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-zhou23f %I PMLR %P 42589--42601 %U https://proceedings.mlr.press/v202/zhou23f.html %V 202 %X Global modeling-based image restoration frameworks have become popular. However, they often require a high memory footprint and do not consider task-specific degradation. Our work presents an alternative approach to global modeling that is more efficient for image restoration. The key insights which motivate our study are two-fold: 1) Fourier transform is capable of disentangling image degradation and content component to a certain extent, serving as the image degradation prior, and 2) Fourier domain innately embraces global properties, where each pixel in the Fourier space is involved with all spatial pixels. While adhering to the “spatial interaction + channel evolution” rule of previous studies, we customize the core designs with Fourier spatial interaction modeling and Fourier channel evolution. Our paradigm, Fourmer, achieves competitive performance on common image restoration tasks such as image de-raining, image enhancement, image dehazing, and guided image super-resolution, while requiring fewer computational resources. The code for Fourmer will be made publicly available.
APA
Zhou, M., Huang, J., Guo, C. & Li, C.. (2023). Fourmer: An Efficient Global Modeling Paradigm for Image Restoration. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:42589-42601 Available from https://proceedings.mlr.press/v202/zhou23f.html.

Related Material