Random Multi-Channel Image Synthesis for Multiplexed Immunofluorescence Imaging

Shunxing Bao, Yucheng Tang, Ho Hin Lee, Riqiang Gao, Sophie Chiron, Ilwoo Lyu, Lori A. Coburn, Keith T. Wilson, Joseph T. Roland, Bennett A. Landman, Yuankai Huo
Proceedings of the MICCAI Workshop on Computational Pathology, PMLR 156:36-46, 2021.

Abstract

Multiplex immunofluorescence (MxIF) is an emerging imaging technique that produces the high sensitivity and specificity of single-cell mapping. With a tenet of “seeing is believing”, MxIF enables iterative staining and imaging extensive antibodies, which provides comprehensive biomarkers to segment and group different cells on a single tissue section. However, considerable depletion of the scarce tissue is inevitable from extensive rounds of staining and bleaching (“missing tissue”). Moreover, the immunofluorescence (IF) imaging can globally fail for particular rounds (“missing stain”). In this work, we focus on the “missing stain” issue. It would be appealing to develop digital image synthesis approaches to restore missing stain images without losing more tissue physically. Herein, we aim to develop image synthesis approaches for eleven MxIF structural molecular markers (i.e., epithelial and stromal) on real samples. We propose a novel multi-channel high-resolution image synthesis approach, called pixN2N-HD, to tackle possible missing stain scenarios via a high-resolution generative adversarial network (GAN). Our contribution is three-fold: (1) a single deep network framework is proposed to tackle missing stain in MxIF; (2) the proposed “N-to-N” strategy reduces theoretical four years of computational time to 20 hours when covering all possible missing stains scenarios, with up to five missing stains (e.g., “(N-1)-to-1”, “(N-2)-to-2”); and (3) this work is the first comprehensive experimental study of investigating cross-stain synthesis in MxIF. Our results elucidate a promising direction of advancing MxIF imaging with deep image synthesis.

Cite this Paper

BibTeX
@InProceedings{pmlr-v156-bao21a, title = {Random Multi-Channel Image Synthesis for Multiplexed Immunofluorescence Imaging}, author = {Bao, Shunxing and Tang, Yucheng and Lee, Ho Hin and Gao, Riqiang and Chiron, Sophie and Lyu, Ilwoo and Coburn, Lori A. and Wilson, Keith T. and Roland, Joseph T. and Landman, Bennett A. and Huo, Yuankai}, booktitle = {Proceedings of the MICCAI Workshop on Computational Pathology}, pages = {36--46}, year = {2021}, editor = {Atzori, Manfredo and Burlutskiy, Nikolay and Ciompi, Francesco and Li, Zhang and Minhas, Fayyaz and Müller, Henning and Peng, Tingying and Rajpoot, Nasir and Torben-Nielsen, Ben and van der Laak, Jeroen and Veta, Mitko and Yuan, Yinyin and Zlobec, Inti}, volume = {156}, series = {Proceedings of Machine Learning Research}, month = {27 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v156/bao21a/bao21a.pdf}, url = {https://proceedings.mlr.press/v156/bao21a.html}, abstract = {Multiplex immunofluorescence (MxIF) is an emerging imaging technique that produces the high sensitivity and specificity of single-cell mapping. With a tenet of “seeing is believing”, MxIF enables iterative staining and imaging extensive antibodies, which provides comprehensive biomarkers to segment and group different cells on a single tissue section. However, considerable depletion of the scarce tissue is inevitable from extensive rounds of staining and bleaching (“missing tissue”). Moreover, the immunofluorescence (IF) imaging can globally fail for particular rounds (“missing stain”). In this work, we focus on the “missing stain” issue. It would be appealing to develop digital image synthesis approaches to restore missing stain images without losing more tissue physically. Herein, we aim to develop image synthesis approaches for eleven MxIF structural molecular markers (i.e., epithelial and stromal) on real samples. We propose a novel multi-channel high-resolution image synthesis approach, called pixN2N-HD, to tackle possible missing stain scenarios via a high-resolution generative adversarial network (GAN). Our contribution is three-fold: (1) a single deep network framework is proposed to tackle missing stain in MxIF; (2) the proposed “N-to-N” strategy reduces theoretical four years of computational time to 20 hours when covering all possible missing stains scenarios, with up to five missing stains (e.g., “(N-1)-to-1”, “(N-2)-to-2”); and (3) this work is the first comprehensive experimental study of investigating cross-stain synthesis in MxIF. Our results elucidate a promising direction of advancing MxIF imaging with deep image synthesis. } }
Endnote
%0 Conference Paper %T Random Multi-Channel Image Synthesis for Multiplexed Immunofluorescence Imaging %A Shunxing Bao %A Yucheng Tang %A Ho Hin Lee %A Riqiang Gao %A Sophie Chiron %A Ilwoo Lyu %A Lori A. Coburn %A Keith T. Wilson %A Joseph T. Roland %A Bennett A. Landman %A Yuankai Huo %B Proceedings of the MICCAI Workshop on Computational Pathology %C Proceedings of Machine Learning Research %D 2021 %E Manfredo Atzori %E Nikolay Burlutskiy %E Francesco Ciompi %E Zhang Li %E Fayyaz Minhas %E Henning Müller %E Tingying Peng %E Nasir Rajpoot %E Ben Torben-Nielsen %E Jeroen van der Laak %E Mitko Veta %E Yinyin Yuan %E Inti Zlobec %F pmlr-v156-bao21a %I PMLR %P 36--46 %U https://proceedings.mlr.press/v156/bao21a.html %V 156 %X Multiplex immunofluorescence (MxIF) is an emerging imaging technique that produces the high sensitivity and specificity of single-cell mapping. With a tenet of “seeing is believing”, MxIF enables iterative staining and imaging extensive antibodies, which provides comprehensive biomarkers to segment and group different cells on a single tissue section. However, considerable depletion of the scarce tissue is inevitable from extensive rounds of staining and bleaching (“missing tissue”). Moreover, the immunofluorescence (IF) imaging can globally fail for particular rounds (“missing stain”). In this work, we focus on the “missing stain” issue. It would be appealing to develop digital image synthesis approaches to restore missing stain images without losing more tissue physically. Herein, we aim to develop image synthesis approaches for eleven MxIF structural molecular markers (i.e., epithelial and stromal) on real samples. We propose a novel multi-channel high-resolution image synthesis approach, called pixN2N-HD, to tackle possible missing stain scenarios via a high-resolution generative adversarial network (GAN). Our contribution is three-fold: (1) a single deep network framework is proposed to tackle missing stain in MxIF; (2) the proposed “N-to-N” strategy reduces theoretical four years of computational time to 20 hours when covering all possible missing stains scenarios, with up to five missing stains (e.g., “(N-1)-to-1”, “(N-2)-to-2”); and (3) this work is the first comprehensive experimental study of investigating cross-stain synthesis in MxIF. Our results elucidate a promising direction of advancing MxIF imaging with deep image synthesis.
APA
Bao, S., Tang, Y., Lee, H.H., Gao, R., Chiron, S., Lyu, I., Coburn, L.A., Wilson, K.T., Roland, J.T., Landman, B.A. & Huo, Y.. (2021). Random Multi-Channel Image Synthesis for Multiplexed Immunofluorescence Imaging. Proceedings of the MICCAI Workshop on Computational Pathology, in Proceedings of Machine Learning Research 156:36-46 Available from https://proceedings.mlr.press/v156/bao21a.html.

Related Material