Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer

Xiangde Luo, Minhao Hu, Tao Song, Guotai Wang, Shaoting Zhang
Proceedings of The 5th International Conference on Medical Imaging with Deep Learning, PMLR 172:820-833, 2022.

Abstract

Recently, deep learning with Convolutional Neural Networks (CNNs) and Transformers has shown encouraging results in fully supervised medical image segmentation. However, it is still challenging for them to achieve good performance with limited annotations for training. This work presents a very simple yet efficient framework for semi-supervised medical image segmentation by introducing the cross teaching between CNN and Transformer. Specifically, we simplify the classical deep co-training from consistency regularization to cross teaching, where the prediction of a network is used as the pseudo label to supervise the other network directly end-to-end. Considering the difference in learning paradigm between CNN and Transformer, we introduce the Cross Teaching between CNN and Transformer rather than just using CNNs. Experiments on a public benchmark show that our method outperforms eight existing semi-supervised learning methods just with a more straight-forward framework. Notably, this work may be the first attempt to combine CNN and transformer for semi-supervised medical image segmentation and achieve promising results on a public benchmark. Code is available at: https://github.com/HiLab-git/SSL4MIS.

Cite this Paper

BibTeX
@InProceedings{pmlr-v172-luo22b, title = {Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer}, author = {Luo, Xiangde and Hu, Minhao and Song, Tao and Wang, Guotai and Zhang, Shaoting}, booktitle = {Proceedings of The 5th International Conference on Medical Imaging with Deep Learning}, pages = {820--833}, year = {2022}, editor = {Konukoglu, Ender and Menze, Bjoern and Venkataraman, Archana and Baumgartner, Christian and Dou, Qi and Albarqouni, Shadi}, volume = {172}, series = {Proceedings of Machine Learning Research}, month = {06--08 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v172/luo22b/luo22b.pdf}, url = {https://proceedings.mlr.press/v172/luo22b.html}, abstract = {Recently, deep learning with Convolutional Neural Networks (CNNs) and Transformers has shown encouraging results in fully supervised medical image segmentation. However, it is still challenging for them to achieve good performance with limited annotations for training. This work presents a very simple yet efficient framework for semi-supervised medical image segmentation by introducing the cross teaching between CNN and Transformer. Specifically, we simplify the classical deep co-training from consistency regularization to cross teaching, where the prediction of a network is used as the pseudo label to supervise the other network directly end-to-end. Considering the difference in learning paradigm between CNN and Transformer, we introduce the Cross Teaching between CNN and Transformer rather than just using CNNs. Experiments on a public benchmark show that our method outperforms eight existing semi-supervised learning methods just with a more straight-forward framework. Notably, this work may be the first attempt to combine CNN and transformer for semi-supervised medical image segmentation and achieve promising results on a public benchmark. Code is available at: https://github.com/HiLab-git/SSL4MIS.} }
Endnote
%0 Conference Paper %T Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer %A Xiangde Luo %A Minhao Hu %A Tao Song %A Guotai Wang %A Shaoting Zhang %B Proceedings of The 5th International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2022 %E Ender Konukoglu %E Bjoern Menze %E Archana Venkataraman %E Christian Baumgartner %E Qi Dou %E Shadi Albarqouni %F pmlr-v172-luo22b %I PMLR %P 820--833 %U https://proceedings.mlr.press/v172/luo22b.html %V 172 %X Recently, deep learning with Convolutional Neural Networks (CNNs) and Transformers has shown encouraging results in fully supervised medical image segmentation. However, it is still challenging for them to achieve good performance with limited annotations for training. This work presents a very simple yet efficient framework for semi-supervised medical image segmentation by introducing the cross teaching between CNN and Transformer. Specifically, we simplify the classical deep co-training from consistency regularization to cross teaching, where the prediction of a network is used as the pseudo label to supervise the other network directly end-to-end. Considering the difference in learning paradigm between CNN and Transformer, we introduce the Cross Teaching between CNN and Transformer rather than just using CNNs. Experiments on a public benchmark show that our method outperforms eight existing semi-supervised learning methods just with a more straight-forward framework. Notably, this work may be the first attempt to combine CNN and transformer for semi-supervised medical image segmentation and achieve promising results on a public benchmark. Code is available at: https://github.com/HiLab-git/SSL4MIS.
APA
Luo, X., Hu, M., Song, T., Wang, G. & Zhang, S.. (2022). Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer. Proceedings of The 5th International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 172:820-833 Available from https://proceedings.mlr.press/v172/luo22b.html.

Related Material