Enhanced Blind Image Restoration with Channel Attention Transformers and Multi-Scale Attention Prompt-based Learning

Jianhua hu, K. L. Eddie Law
Proceedings of the 17th Asian Conference on Machine Learning, PMLR 304:367-382, 2025.

Abstract

Deep learning models today are indispensable tools for image compression and restoration. However, despite recent progress, many existing models often lack generalization upon facing with different types and coding strength designs of image restoration, thus limiting their practical application. In this paper, a novel approach called \{\\em dual-Channel Transformers and Multi-scale attention Prompt learning (CTMP)\} is introduced to bridge the gap on blind image restoration. The prompt-based learning approach is employed in the model to address two key image restoration tasks: 1) compressed image artifact removal, and 2) image denoising. By utilizing adaptive prompts to accommodate varying quantization parameter (QP) values and noise conditions, and enhancing adaptability through the integration of multi-scale attention mechanisms, an advanced Transformer architecture in our model can tackle diverse image degradations in blind image restoration. That is, our Transformer module is improved through merging and harnessing the strengths of both channel attention and self-attention. The design is adept at extracting both high-frequency details and low-frequency structures, thereby significantly enhancing overall restoration performance. Using the Kodak dataset in experiments, our model outperforms conventional deep learning techniques with a 2.44% reduction of BD-rate in blind mode. It shows a 29.21% improvement over traditional JPEG compression and a 0.14 dB improvement in blind denoising. The experiments demonstrate that our approach is capable of training a single model effectively for both compressed image artifact removal and image denoising. The code is publicly available on GitHub at https://github.com/gdit-ai/CTMP.

Cite this Paper

BibTeX
@InProceedings{pmlr-v304-hu25a, title = {Enhanced Blind Image Restoration with Channel Attention Transformers and Multi-Scale Attention Prompt-based Learning}, author = {hu, Jianhua and Law, K. L. Eddie}, booktitle = {Proceedings of the 17th Asian Conference on Machine Learning}, pages = {367--382}, year = {2025}, editor = {Lee, Hung-yi and Liu, Tongliang}, volume = {304}, series = {Proceedings of Machine Learning Research}, month = {09--12 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v304/main/assets/hu25a/hu25a.pdf}, url = {https://proceedings.mlr.press/v304/hu25a.html}, abstract = {Deep learning models today are indispensable tools for image compression and restoration. However, despite recent progress, many existing models often lack generalization upon facing with different types and coding strength designs of image restoration, thus limiting their practical application. In this paper, a novel approach called \{\\em dual-Channel Transformers and Multi-scale attention Prompt learning (CTMP)\} is introduced to bridge the gap on blind image restoration. The prompt-based learning approach is employed in the model to address two key image restoration tasks: 1) compressed image artifact removal, and 2) image denoising. By utilizing adaptive prompts to accommodate varying quantization parameter (QP) values and noise conditions, and enhancing adaptability through the integration of multi-scale attention mechanisms, an advanced Transformer architecture in our model can tackle diverse image degradations in blind image restoration. That is, our Transformer module is improved through merging and harnessing the strengths of both channel attention and self-attention. The design is adept at extracting both high-frequency details and low-frequency structures, thereby significantly enhancing overall restoration performance. Using the Kodak dataset in experiments, our model outperforms conventional deep learning techniques with a 2.44% reduction of BD-rate in blind mode. It shows a 29.21% improvement over traditional JPEG compression and a 0.14 dB improvement in blind denoising. The experiments demonstrate that our approach is capable of training a single model effectively for both compressed image artifact removal and image denoising. The code is publicly available on GitHub at https://github.com/gdit-ai/CTMP.} }
Endnote
%0 Conference Paper %T Enhanced Blind Image Restoration with Channel Attention Transformers and Multi-Scale Attention Prompt-based Learning %A Jianhua hu %A K. L. Eddie Law %B Proceedings of the 17th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Hung-yi Lee %E Tongliang Liu %F pmlr-v304-hu25a %I PMLR %P 367--382 %U https://proceedings.mlr.press/v304/hu25a.html %V 304 %X Deep learning models today are indispensable tools for image compression and restoration. However, despite recent progress, many existing models often lack generalization upon facing with different types and coding strength designs of image restoration, thus limiting their practical application. In this paper, a novel approach called \{\\em dual-Channel Transformers and Multi-scale attention Prompt learning (CTMP)\} is introduced to bridge the gap on blind image restoration. The prompt-based learning approach is employed in the model to address two key image restoration tasks: 1) compressed image artifact removal, and 2) image denoising. By utilizing adaptive prompts to accommodate varying quantization parameter (QP) values and noise conditions, and enhancing adaptability through the integration of multi-scale attention mechanisms, an advanced Transformer architecture in our model can tackle diverse image degradations in blind image restoration. That is, our Transformer module is improved through merging and harnessing the strengths of both channel attention and self-attention. The design is adept at extracting both high-frequency details and low-frequency structures, thereby significantly enhancing overall restoration performance. Using the Kodak dataset in experiments, our model outperforms conventional deep learning techniques with a 2.44% reduction of BD-rate in blind mode. It shows a 29.21% improvement over traditional JPEG compression and a 0.14 dB improvement in blind denoising. The experiments demonstrate that our approach is capable of training a single model effectively for both compressed image artifact removal and image denoising. The code is publicly available on GitHub at https://github.com/gdit-ai/CTMP.
APA
hu, J. & Law, K.L.E.. (2025). Enhanced Blind Image Restoration with Channel Attention Transformers and Multi-Scale Attention Prompt-based Learning. Proceedings of the 17th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 304:367-382 Available from https://proceedings.mlr.press/v304/hu25a.html.

Related Material