MR Image Super Resolution By Combining Feature Disentanglement CNNs and Vision Transformers

Dwarikanath Mahapatra, Zongyuan Ge
Proceedings of The 5th International Conference on Medical Imaging with Deep Learning, PMLR 172:858-878, 2022.

Abstract

State of the art magnetic resonance (MR) image super-resolution methods (ISR) using convolutional neural networks (CNNs) leverage limited contextual information due to the limited spatial coverage of CNNs. Vision transformers (ViT) learn better global context that is helpful in generating superior quality HR images. We combine local information of CNNs and global information from ViTs for image super resolution and output super resolved images that have superior quality than those produced by state of the art methods. We include extra constraints through multiple novel loss functions that preserve structure and texture information from the low resolution to high resolution images.

Cite this Paper

BibTeX
@InProceedings{pmlr-v172-mahapatra22a, title = {MR Image Super Resolution By Combining Feature Disentanglement CNNs and Vision Transformers}, author = {Mahapatra, Dwarikanath and Ge, Zongyuan}, booktitle = {Proceedings of The 5th International Conference on Medical Imaging with Deep Learning}, pages = {858--878}, year = {2022}, editor = {Konukoglu, Ender and Menze, Bjoern and Venkataraman, Archana and Baumgartner, Christian and Dou, Qi and Albarqouni, Shadi}, volume = {172}, series = {Proceedings of Machine Learning Research}, month = {06--08 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v172/mahapatra22a/mahapatra22a.pdf}, url = {https://proceedings.mlr.press/v172/mahapatra22a.html}, abstract = {State of the art magnetic resonance (MR) image super-resolution methods (ISR) using convolutional neural networks (CNNs) leverage limited contextual information due to the limited spatial coverage of CNNs. Vision transformers (ViT) learn better global context that is helpful in generating superior quality HR images. We combine local information of CNNs and global information from ViTs for image super resolution and output super resolved images that have superior quality than those produced by state of the art methods. We include extra constraints through multiple novel loss functions that preserve structure and texture information from the low resolution to high resolution images.} }
Endnote
%0 Conference Paper %T MR Image Super Resolution By Combining Feature Disentanglement CNNs and Vision Transformers %A Dwarikanath Mahapatra %A Zongyuan Ge %B Proceedings of The 5th International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2022 %E Ender Konukoglu %E Bjoern Menze %E Archana Venkataraman %E Christian Baumgartner %E Qi Dou %E Shadi Albarqouni %F pmlr-v172-mahapatra22a %I PMLR %P 858--878 %U https://proceedings.mlr.press/v172/mahapatra22a.html %V 172 %X State of the art magnetic resonance (MR) image super-resolution methods (ISR) using convolutional neural networks (CNNs) leverage limited contextual information due to the limited spatial coverage of CNNs. Vision transformers (ViT) learn better global context that is helpful in generating superior quality HR images. We combine local information of CNNs and global information from ViTs for image super resolution and output super resolved images that have superior quality than those produced by state of the art methods. We include extra constraints through multiple novel loss functions that preserve structure and texture information from the low resolution to high resolution images.
APA
Mahapatra, D. & Ge, Z.. (2022). MR Image Super Resolution By Combining Feature Disentanglement CNNs and Vision Transformers. Proceedings of The 5th International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 172:858-878 Available from https://proceedings.mlr.press/v172/mahapatra22a.html.

Related Material