Graph Neural Networks Ameliorate Potential Impacts of Imprecise Large-Scale Autonomous Immunofluorescence Labeling of Immune Cells on Whole Slide Images

Ramya Reddy, Ram Reddy, Cyril Sharma, Christopher Jackson, Scott Palisoul, Rachael Barney, Fred Kolling, Lucas Salas, Brock Christensen, Gabriel Brooks, Gregory Tsongalis, Louis Vaickus, Joshua Levy
Proceedings of the First International Workshop on Geometric Deep Learning in Medical Image Analysis, PMLR 194:15-33, 2022.

Abstract

The characteristics of tumor-infiltrating lymphocytes (TILs) are essential for cancer prognostication and treatment through the ability to indicate the tumor’s capacity to evade the immune system (e.g., as evidenced by nodal involvement). In general, presence of TILs indicates a favorable prognosis. Machine learning technologies have demonstrated remarkable success for localizing TILs, though these methods require extensive curation of manual annotations or restaining procedures that can degrade tissue quality, resulting in imprecise annotation. In this study, we co-registered tissue slides stained for both hematoxylin and eosin (H&E) and immunofluorescence (IF) as means to rapidly perform large-scale annotation of nuclei. We integrated the following approaches to improve the prediction of TILs: 1) minimized tissue degradation on same-section tissue restaining, 2) developed a scoring algorithm to improve the selection of patches for machine learning modeling and 3) utilized a graph neural network deep learning approach to identify relevant contextual features for lymphocyte prediction. Our graph neural network approach accounts for surrounding contextual micro/macro-architecture tissue features to facilitate interpretation of registered IF. The graph neural network compares favorably (F1-score=0.9235, AUROC=0.9462) to two alternative modeling approaches. This study brings insight to the importance of contextual information leveraged from within and around neighboring cells in a nuclei classification workflow, as well as elucidate approaches which enable the rapid generation of large-scale annotations of lymphocytes for machine learning approaches for immune phenotyping. Such approaches can help further interrogate the spatial biology of colorectal cancer tumors and tumor metastasis.

Cite this Paper

BibTeX
@InProceedings{pmlr-v194-reddy22a, title = {Graph Neural Networks Ameliorate Potential Impacts of Imprecise Large-Scale Autonomous Immunofluorescence Labeling of Immune Cells on Whole Slide Images}, author = {Reddy, Ramya and Reddy, Ram and Sharma, Cyril and Jackson, Christopher and Palisoul, Scott and Barney, Rachael and Kolling, Fred and Salas, Lucas and Christensen, Brock and Brooks, Gabriel and Tsongalis, Gregory and Vaickus, Louis and Levy, Joshua}, booktitle = {Proceedings of the First International Workshop on Geometric Deep Learning in Medical Image Analysis}, pages = {15--33}, year = {2022}, editor = {Bekkers, Erik and Wolterink, Jelmer M. and Aviles-Rivero, Angelica}, volume = {194}, series = {Proceedings of Machine Learning Research}, month = {18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v194/reddy22a/reddy22a.pdf}, url = {https://proceedings.mlr.press/v194/reddy22a.html}, abstract = {The characteristics of tumor-infiltrating lymphocytes (TILs) are essential for cancer prognostication and treatment through the ability to indicate the tumor’s capacity to evade the immune system (e.g., as evidenced by nodal involvement). In general, presence of TILs indicates a favorable prognosis. Machine learning technologies have demonstrated remarkable success for localizing TILs, though these methods require extensive curation of manual annotations or restaining procedures that can degrade tissue quality, resulting in imprecise annotation. In this study, we co-registered tissue slides stained for both hematoxylin and eosin (H&E) and immunofluorescence (IF) as means to rapidly perform large-scale annotation of nuclei. We integrated the following approaches to improve the prediction of TILs: 1) minimized tissue degradation on same-section tissue restaining, 2) developed a scoring algorithm to improve the selection of patches for machine learning modeling and 3) utilized a graph neural network deep learning approach to identify relevant contextual features for lymphocyte prediction. Our graph neural network approach accounts for surrounding contextual micro/macro-architecture tissue features to facilitate interpretation of registered IF. The graph neural network compares favorably (F1-score=0.9235, AUROC=0.9462) to two alternative modeling approaches. This study brings insight to the importance of contextual information leveraged from within and around neighboring cells in a nuclei classification workflow, as well as elucidate approaches which enable the rapid generation of large-scale annotations of lymphocytes for machine learning approaches for immune phenotyping. Such approaches can help further interrogate the spatial biology of colorectal cancer tumors and tumor metastasis. } }
Endnote
%0 Conference Paper %T Graph Neural Networks Ameliorate Potential Impacts of Imprecise Large-Scale Autonomous Immunofluorescence Labeling of Immune Cells on Whole Slide Images %A Ramya Reddy %A Ram Reddy %A Cyril Sharma %A Christopher Jackson %A Scott Palisoul %A Rachael Barney %A Fred Kolling %A Lucas Salas %A Brock Christensen %A Gabriel Brooks %A Gregory Tsongalis %A Louis Vaickus %A Joshua Levy %B Proceedings of the First International Workshop on Geometric Deep Learning in Medical Image Analysis %C Proceedings of Machine Learning Research %D 2022 %E Erik Bekkers %E Jelmer M. Wolterink %E Angelica Aviles-Rivero %F pmlr-v194-reddy22a %I PMLR %P 15--33 %U https://proceedings.mlr.press/v194/reddy22a.html %V 194 %X The characteristics of tumor-infiltrating lymphocytes (TILs) are essential for cancer prognostication and treatment through the ability to indicate the tumor’s capacity to evade the immune system (e.g., as evidenced by nodal involvement). In general, presence of TILs indicates a favorable prognosis. Machine learning technologies have demonstrated remarkable success for localizing TILs, though these methods require extensive curation of manual annotations or restaining procedures that can degrade tissue quality, resulting in imprecise annotation. In this study, we co-registered tissue slides stained for both hematoxylin and eosin (H&E) and immunofluorescence (IF) as means to rapidly perform large-scale annotation of nuclei. We integrated the following approaches to improve the prediction of TILs: 1) minimized tissue degradation on same-section tissue restaining, 2) developed a scoring algorithm to improve the selection of patches for machine learning modeling and 3) utilized a graph neural network deep learning approach to identify relevant contextual features for lymphocyte prediction. Our graph neural network approach accounts for surrounding contextual micro/macro-architecture tissue features to facilitate interpretation of registered IF. The graph neural network compares favorably (F1-score=0.9235, AUROC=0.9462) to two alternative modeling approaches. This study brings insight to the importance of contextual information leveraged from within and around neighboring cells in a nuclei classification workflow, as well as elucidate approaches which enable the rapid generation of large-scale annotations of lymphocytes for machine learning approaches for immune phenotyping. Such approaches can help further interrogate the spatial biology of colorectal cancer tumors and tumor metastasis.
APA
Reddy, R., Reddy, R., Sharma, C., Jackson, C., Palisoul, S., Barney, R., Kolling, F., Salas, L., Christensen, B., Brooks, G., Tsongalis, G., Vaickus, L. & Levy, J.. (2022). Graph Neural Networks Ameliorate Potential Impacts of Imprecise Large-Scale Autonomous Immunofluorescence Labeling of Immune Cells on Whole Slide Images. Proceedings of the First International Workshop on Geometric Deep Learning in Medical Image Analysis, in Proceedings of Machine Learning Research 194:15-33 Available from https://proceedings.mlr.press/v194/reddy22a.html.

Related Material