Early Fusion of H&E and IHC Histology Images for Pediatric Brain Tumor Classification

Christoforos Spyretos, Iulian Emil Tampu, Nadieh Khalili, Juan Manuel Pardo Ladino, Per Nyman, Ida Blystad, Anders Eklund, Neda Haj-Hosseini
Proceedings of the MICCAI Workshop on Computational Pathology, PMLR 254:192-202, 2024.

Abstract

This study explores the application of computational pathology to analyze pediatric brain tumors utilizing hematoxylin and eosin (H&E) and immunohistochemistry (IHC) whole slide images (WSIs). Experiments were conducted on H&E images for predicting tumor diagnosis and fusing them with unregistered IHC images to investigate potential improvements. Patch features were extracted using UNI, a vision transformer (ViT) model trained on H&E data, and whole slide classification was achieved using the attention-based multiple instance learning CLAM framework. In the astrocytoma tumor classification, early fusion of the H&E and IHC significantly improved the differentiation between tumor grades (balanced accuracy: 0.82 ± 0.05 vs 0.84 ± 0.05). In the multiclass classification, H&E images alone had a balanced accuracy of 0.79 ± 0.03 without any improvement obtained when fused with IHC. The findings highlight the potential of using multi-stain fusion to advance the diagnosis of pediatric brain tumors, however, further fusion methods should be investigated.

Cite this Paper

BibTeX
@InProceedings{pmlr-v254-spyretos24a, title = {Early Fusion of H&E and IHC Histology Images for Pediatric Brain Tumor Classification}, author = {Spyretos, Christoforos and Tampu, Iulian Emil and Khalili, Nadieh and Ladino, Juan Manuel Pardo and Nyman, Per and Blystad, Ida and Eklund, Anders and Haj-Hosseini, Neda}, booktitle = {Proceedings of the MICCAI Workshop on Computational Pathology}, pages = {192--202}, year = {2024}, editor = {Ciompi, Francesco and Khalili, Nadieh and Studer, Linda and Poceviciute, Milda and Khan, Amjad and Veta, Mitko and Jiao, Yiping and Haj-Hosseini, Neda and Chen, Hao and Raza, Shan and Minhas, FayyazZlobec, Inti and Burlutskiy, Nikolay and Vilaplana, Veronica and Brattoli, Biagio and Muller, Henning and Atzori, Manfredo and Raza, Shan and Minhas, Fayyaz}, volume = {254}, series = {Proceedings of Machine Learning Research}, month = {06 Oct}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v254/main/assets/spyretos24a/spyretos24a.pdf}, url = {https://proceedings.mlr.press/v254/spyretos24a.html}, abstract = {This study explores the application of computational pathology to analyze pediatric brain tumors utilizing hematoxylin and eosin (H&E) and immunohistochemistry (IHC) whole slide images (WSIs). Experiments were conducted on H&E images for predicting tumor diagnosis and fusing them with unregistered IHC images to investigate potential improvements. Patch features were extracted using UNI, a vision transformer (ViT) model trained on H&E data, and whole slide classification was achieved using the attention-based multiple instance learning CLAM framework. In the astrocytoma tumor classification, early fusion of the H&E and IHC significantly improved the differentiation between tumor grades (balanced accuracy: 0.82 ± 0.05 vs 0.84 ± 0.05). In the multiclass classification, H&E images alone had a balanced accuracy of 0.79 ± 0.03 without any improvement obtained when fused with IHC. The findings highlight the potential of using multi-stain fusion to advance the diagnosis of pediatric brain tumors, however, further fusion methods should be investigated.} }
Endnote
%0 Conference Paper %T Early Fusion of H&E and IHC Histology Images for Pediatric Brain Tumor Classification %A Christoforos Spyretos %A Iulian Emil Tampu %A Nadieh Khalili %A Juan Manuel Pardo Ladino %A Per Nyman %A Ida Blystad %A Anders Eklund %A Neda Haj-Hosseini %B Proceedings of the MICCAI Workshop on Computational Pathology %C Proceedings of Machine Learning Research %D 2024 %E Francesco Ciompi %E Nadieh Khalili %E Linda Studer %E Milda Poceviciute %E Amjad Khan %E Mitko Veta %E Yiping Jiao %E Neda Haj-Hosseini %E Hao Chen %E Shan Raza %E Fayyaz MinhasInti Zlobec %E Nikolay Burlutskiy %E Veronica Vilaplana %E Biagio Brattoli %E Henning Muller %E Manfredo Atzori %E Shan Raza %E Fayyaz Minhas %F pmlr-v254-spyretos24a %I PMLR %P 192--202 %U https://proceedings.mlr.press/v254/spyretos24a.html %V 254 %X This study explores the application of computational pathology to analyze pediatric brain tumors utilizing hematoxylin and eosin (H&E) and immunohistochemistry (IHC) whole slide images (WSIs). Experiments were conducted on H&E images for predicting tumor diagnosis and fusing them with unregistered IHC images to investigate potential improvements. Patch features were extracted using UNI, a vision transformer (ViT) model trained on H&E data, and whole slide classification was achieved using the attention-based multiple instance learning CLAM framework. In the astrocytoma tumor classification, early fusion of the H&E and IHC significantly improved the differentiation between tumor grades (balanced accuracy: 0.82 ± 0.05 vs 0.84 ± 0.05). In the multiclass classification, H&E images alone had a balanced accuracy of 0.79 ± 0.03 without any improvement obtained when fused with IHC. The findings highlight the potential of using multi-stain fusion to advance the diagnosis of pediatric brain tumors, however, further fusion methods should be investigated.
APA
Spyretos, C., Tampu, I.E., Khalili, N., Ladino, J.M.P., Nyman, P., Blystad, I., Eklund, A. & Haj-Hosseini, N.. (2024). Early Fusion of H&E and IHC Histology Images for Pediatric Brain Tumor Classification. Proceedings of the MICCAI Workshop on Computational Pathology, in Proceedings of Machine Learning Research 254:192-202 Available from https://proceedings.mlr.press/v254/spyretos24a.html.

Related Material