Deep-Learning Based Virtual Stain Multiplexing Immunohistochemistry Slides – a Pilot Study

Oded Ben-David, Elad Arbel, Daniela Rabkin, Itay Remer, Amir Ben-Dor, Sarit Aviel-Ronen, Frederik Aidt, Tine Hagedorn-Olsen, Lars Jacobsen, Kristopher Kersch, Anya Tsalenko
Proceedings of the MICCAI Workshop on Computational Pathology, PMLR 254:107-120, 2024.

Abstract

In this paper, we introduce a novel deep-learning based method for virtual stain multiplexing of immunohistochemistry (IHC) stains. Traditional IHC techniques generally involve a single stain that highlights a single target protein, but this can be enriched with stain multiplexing. Our proposed method leverages sequential staining to train a model to virtually stain multiplex additional IHC on top of a digitally scanned whole slide image (WSI), without requiring a complex setup or any additional tissue sections and stains. To this end, we designed a novel model architecture, guided by the physical sequential staining process which provides superior performance. The model was optimized using a custom loss function that combines mean squared error (MSE) with semantic information, allowing the model to focus on learning the relevant differences between the input and ground truth. As an example application, we consider the problem of detecting macro-phages on PD-L1 IHC 22C3 pharmDx NSCLC WSIs. We demonstrated virtual stain multiplexing CD68 on top of PD-L1 22C3 pharmDx stained slides, which helps to detect macrophages and distinguish them from PD-L1+ tumor cells, which are often visually similar. Our pilot-study results showed significant improvement in a pathologist’s ability to distinguish macrophages when using the virtually stain multiplexed CD68 decision supporting layer.

Cite this Paper

BibTeX
@InProceedings{pmlr-v254-ben-david24a, title = {Deep-Learning Based Virtual Stain Multiplexing Immunohistochemistry Slides – a Pilot Study}, author = {Ben-David, Oded and Arbel, Elad and Rabkin, Daniela and Remer, Itay and Ben-Dor, Amir and Aviel-Ronen, Sarit and Aidt, Frederik and Hagedorn-Olsen, Tine and Jacobsen, Lars and Kersch, Kristopher and Tsalenko, Anya}, booktitle = {Proceedings of the MICCAI Workshop on Computational Pathology}, pages = {107--120}, year = {2024}, editor = {Ciompi, Francesco and Khalili, Nadieh and Studer, Linda and Poceviciute, Milda and Khan, Amjad and Veta, Mitko and Jiao, Yiping and Haj-Hosseini, Neda and Chen, Hao and Raza, Shan and Minhas, FayyazZlobec, Inti and Burlutskiy, Nikolay and Vilaplana, Veronica and Brattoli, Biagio and Muller, Henning and Atzori, Manfredo and Raza, Shan and Minhas, Fayyaz}, volume = {254}, series = {Proceedings of Machine Learning Research}, month = {06 Oct}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v254/main/assets/ben-david24a/ben-david24a.pdf}, url = {https://proceedings.mlr.press/v254/ben-david24a.html}, abstract = {In this paper, we introduce a novel deep-learning based method for virtual stain multiplexing of immunohistochemistry (IHC) stains. Traditional IHC techniques generally involve a single stain that highlights a single target protein, but this can be enriched with stain multiplexing. Our proposed method leverages sequential staining to train a model to virtually stain multiplex additional IHC on top of a digitally scanned whole slide image (WSI), without requiring a complex setup or any additional tissue sections and stains. To this end, we designed a novel model architecture, guided by the physical sequential staining process which provides superior performance. The model was optimized using a custom loss function that combines mean squared error (MSE) with semantic information, allowing the model to focus on learning the relevant differences between the input and ground truth. As an example application, we consider the problem of detecting macro-phages on PD-L1 IHC 22C3 pharmDx NSCLC WSIs. We demonstrated virtual stain multiplexing CD68 on top of PD-L1 22C3 pharmDx stained slides, which helps to detect macrophages and distinguish them from PD-L1+ tumor cells, which are often visually similar. Our pilot-study results showed significant improvement in a pathologist’s ability to distinguish macrophages when using the virtually stain multiplexed CD68 decision supporting layer.} }
Endnote
%0 Conference Paper %T Deep-Learning Based Virtual Stain Multiplexing Immunohistochemistry Slides – a Pilot Study %A Oded Ben-David %A Elad Arbel %A Daniela Rabkin %A Itay Remer %A Amir Ben-Dor %A Sarit Aviel-Ronen %A Frederik Aidt %A Tine Hagedorn-Olsen %A Lars Jacobsen %A Kristopher Kersch %A Anya Tsalenko %B Proceedings of the MICCAI Workshop on Computational Pathology %C Proceedings of Machine Learning Research %D 2024 %E Francesco Ciompi %E Nadieh Khalili %E Linda Studer %E Milda Poceviciute %E Amjad Khan %E Mitko Veta %E Yiping Jiao %E Neda Haj-Hosseini %E Hao Chen %E Shan Raza %E Fayyaz MinhasInti Zlobec %E Nikolay Burlutskiy %E Veronica Vilaplana %E Biagio Brattoli %E Henning Muller %E Manfredo Atzori %E Shan Raza %E Fayyaz Minhas %F pmlr-v254-ben-david24a %I PMLR %P 107--120 %U https://proceedings.mlr.press/v254/ben-david24a.html %V 254 %X In this paper, we introduce a novel deep-learning based method for virtual stain multiplexing of immunohistochemistry (IHC) stains. Traditional IHC techniques generally involve a single stain that highlights a single target protein, but this can be enriched with stain multiplexing. Our proposed method leverages sequential staining to train a model to virtually stain multiplex additional IHC on top of a digitally scanned whole slide image (WSI), without requiring a complex setup or any additional tissue sections and stains. To this end, we designed a novel model architecture, guided by the physical sequential staining process which provides superior performance. The model was optimized using a custom loss function that combines mean squared error (MSE) with semantic information, allowing the model to focus on learning the relevant differences between the input and ground truth. As an example application, we consider the problem of detecting macro-phages on PD-L1 IHC 22C3 pharmDx NSCLC WSIs. We demonstrated virtual stain multiplexing CD68 on top of PD-L1 22C3 pharmDx stained slides, which helps to detect macrophages and distinguish them from PD-L1+ tumor cells, which are often visually similar. Our pilot-study results showed significant improvement in a pathologist’s ability to distinguish macrophages when using the virtually stain multiplexed CD68 decision supporting layer.
APA
Ben-David, O., Arbel, E., Rabkin, D., Remer, I., Ben-Dor, A., Aviel-Ronen, S., Aidt, F., Hagedorn-Olsen, T., Jacobsen, L., Kersch, K. & Tsalenko, A.. (2024). Deep-Learning Based Virtual Stain Multiplexing Immunohistochemistry Slides – a Pilot Study. Proceedings of the MICCAI Workshop on Computational Pathology, in Proceedings of Machine Learning Research 254:107-120 Available from https://proceedings.mlr.press/v254/ben-david24a.html.

Related Material