Histopathobiome – integrating histopathology and microbiome data via multimodal deep learning

Agata Polejowska, Annemarie Boleij, Francesco Ciompi
Proceedings of the MICCAI Workshop on Computational Pathology, PMLR 254:203-213, 2024.

Abstract

We introduce Histopathobiome, a term representing the integration of histopathology and microbiome data to explore tissue-microbe interactions. Using a dataset of colon biopsy whole-slide images paired with microbiota composition samples, we assess the benefits of combining these modalities to distinguish patients with inflammatory bowel disease (IBD) subtype – ulcerative colitis (UC) from non-IBD controls. Initially, we evaluate the unimodal performance of state-of-the-art algorithms using vectors representing bacterial species abundances or histopathology slide-level embeddings. We compare single-modality models with bimodal networks with various fusion strategies. Our results prove that histopathology and microbiome data are complementary in UC classification. By demonstrating improved performance over single-modality approaches, we prove that bimodal deep learning models can be used to learn meaningful and interpretable cross-modal tissue-microbe patterns.

Cite this Paper

BibTeX
@InProceedings{pmlr-v254-polejowska24a, title = {Histopathobiome – integrating histopathology and microbiome data via multimodal deep learning}, author = {Polejowska, Agata and Boleij, Annemarie and Ciompi, Francesco}, booktitle = {Proceedings of the MICCAI Workshop on Computational Pathology}, pages = {203--213}, year = {2024}, editor = {Ciompi, Francesco and Khalili, Nadieh and Studer, Linda and Poceviciute, Milda and Khan, Amjad and Veta, Mitko and Jiao, Yiping and Haj-Hosseini, Neda and Chen, Hao and Raza, Shan and Minhas, FayyazZlobec, Inti and Burlutskiy, Nikolay and Vilaplana, Veronica and Brattoli, Biagio and Muller, Henning and Atzori, Manfredo and Raza, Shan and Minhas, Fayyaz}, volume = {254}, series = {Proceedings of Machine Learning Research}, month = {06 Oct}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v254/main/assets/polejowska24a/polejowska24a.pdf}, url = {https://proceedings.mlr.press/v254/polejowska24a.html}, abstract = {We introduce Histopathobiome, a term representing the integration of histopathology and microbiome data to explore tissue-microbe interactions. Using a dataset of colon biopsy whole-slide images paired with microbiota composition samples, we assess the benefits of combining these modalities to distinguish patients with inflammatory bowel disease (IBD) subtype – ulcerative colitis (UC) from non-IBD controls. Initially, we evaluate the unimodal performance of state-of-the-art algorithms using vectors representing bacterial species abundances or histopathology slide-level embeddings. We compare single-modality models with bimodal networks with various fusion strategies. Our results prove that histopathology and microbiome data are complementary in UC classification. By demonstrating improved performance over single-modality approaches, we prove that bimodal deep learning models can be used to learn meaningful and interpretable cross-modal tissue-microbe patterns.} }
Endnote
%0 Conference Paper %T Histopathobiome – integrating histopathology and microbiome data via multimodal deep learning %A Agata Polejowska %A Annemarie Boleij %A Francesco Ciompi %B Proceedings of the MICCAI Workshop on Computational Pathology %C Proceedings of Machine Learning Research %D 2024 %E Francesco Ciompi %E Nadieh Khalili %E Linda Studer %E Milda Poceviciute %E Amjad Khan %E Mitko Veta %E Yiping Jiao %E Neda Haj-Hosseini %E Hao Chen %E Shan Raza %E Fayyaz MinhasInti Zlobec %E Nikolay Burlutskiy %E Veronica Vilaplana %E Biagio Brattoli %E Henning Muller %E Manfredo Atzori %E Shan Raza %E Fayyaz Minhas %F pmlr-v254-polejowska24a %I PMLR %P 203--213 %U https://proceedings.mlr.press/v254/polejowska24a.html %V 254 %X We introduce Histopathobiome, a term representing the integration of histopathology and microbiome data to explore tissue-microbe interactions. Using a dataset of colon biopsy whole-slide images paired with microbiota composition samples, we assess the benefits of combining these modalities to distinguish patients with inflammatory bowel disease (IBD) subtype – ulcerative colitis (UC) from non-IBD controls. Initially, we evaluate the unimodal performance of state-of-the-art algorithms using vectors representing bacterial species abundances or histopathology slide-level embeddings. We compare single-modality models with bimodal networks with various fusion strategies. Our results prove that histopathology and microbiome data are complementary in UC classification. By demonstrating improved performance over single-modality approaches, we prove that bimodal deep learning models can be used to learn meaningful and interpretable cross-modal tissue-microbe patterns.
APA
Polejowska, A., Boleij, A. & Ciompi, F.. (2024). Histopathobiome – integrating histopathology and microbiome data via multimodal deep learning. Proceedings of the MICCAI Workshop on Computational Pathology, in Proceedings of Machine Learning Research 254:203-213 Available from https://proceedings.mlr.press/v254/polejowska24a.html.

Related Material