Multi-Scale Task Multiple Instance Learning for the Classification of Digital Pathology Images with Global Annotations

Niccolò Marini, Sebastian Otálora, Francesco Ciompi, Gianmaria Silvello, Stefano Marchesin, Simona Vatrano, Genziana Buttafuoco, Manfredo Atzori, Henning Müller
Proceedings of the MICCAI Workshop on Computational Pathology, PMLR 156:170-181, 2021.

Abstract

Whole slide images (WSIs) are high-resolution digitized images of tissue samples, stored including different magnification levels. WSIs datasets often include only global annotations, available thanks to pathology reports. Global annotations refer to global findings in the high-resolution image and do not include information about the location of the regions of interest or the magnification levels used to identify a finding. This fact can limit the training of machine learning models, as WSIs are usually very large and each magnification level includes different information about the tissue. This paper presents a Multi-Scale Task Multiple Instance Learning (MuSTMIL) method, allowing to better exploit data paired with global labels and to combine contextual and detailed information identified at several magnification levels. The method is based on a multiple instance learning framework and on a multi-task network, that combines features from several magnification levels and produces multiple predictions (a global one and one for each magnification level involved). MuSTMIL is evaluated on colon cancer images, on binary and multilabel classification. MuSTMIL shows an improvement in performance in comparison to both single scale and another multi-scale multiple instance learning algorithm, demonstrating that MuSTMIL can help to better deal with global labels targeting full and multi-scale images.

Cite this Paper

BibTeX
@InProceedings{pmlr-v156-marini21a, title = {Multi-Scale Task Multiple Instance Learning for the Classification of Digital Pathology Images with Global Annotations}, author = {Marini, Niccol{\`o} and Ot{\'a}lora, Sebastian and Ciompi, Francesco and Silvello, Gianmaria and Marchesin, Stefano and Vatrano, Simona and Buttafuoco, Genziana and Atzori, Manfredo and M{\"u}ller, Henning}, booktitle = {Proceedings of the MICCAI Workshop on Computational Pathology}, pages = {170--181}, year = {2021}, editor = {Atzori, Manfredo and Burlutskiy, Nikolay and Ciompi, Francesco and Li, Zhang and Minhas, Fayyaz and Müller, Henning and Peng, Tingying and Rajpoot, Nasir and Torben-Nielsen, Ben and van der Laak, Jeroen and Veta, Mitko and Yuan, Yinyin and Zlobec, Inti}, volume = {156}, series = {Proceedings of Machine Learning Research}, month = {27 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v156/marini21a/marini21a.pdf}, url = {https://proceedings.mlr.press/v156/marini21a.html}, abstract = {Whole slide images (WSIs) are high-resolution digitized images of tissue samples, stored including different magnification levels. WSIs datasets often include only global annotations, available thanks to pathology reports. Global annotations refer to global findings in the high-resolution image and do not include information about the location of the regions of interest or the magnification levels used to identify a finding. This fact can limit the training of machine learning models, as WSIs are usually very large and each magnification level includes different information about the tissue. This paper presents a Multi-Scale Task Multiple Instance Learning (MuSTMIL) method, allowing to better exploit data paired with global labels and to combine contextual and detailed information identified at several magnification levels. The method is based on a multiple instance learning framework and on a multi-task network, that combines features from several magnification levels and produces multiple predictions (a global one and one for each magnification level involved). MuSTMIL is evaluated on colon cancer images, on binary and multilabel classification. MuSTMIL shows an improvement in performance in comparison to both single scale and another multi-scale multiple instance learning algorithm, demonstrating that MuSTMIL can help to better deal with global labels targeting full and multi-scale images.} }
Endnote
%0 Conference Paper %T Multi-Scale Task Multiple Instance Learning for the Classification of Digital Pathology Images with Global Annotations %A Niccolò Marini %A Sebastian Otálora %A Francesco Ciompi %A Gianmaria Silvello %A Stefano Marchesin %A Simona Vatrano %A Genziana Buttafuoco %A Manfredo Atzori %A Henning Müller %B Proceedings of the MICCAI Workshop on Computational Pathology %C Proceedings of Machine Learning Research %D 2021 %E Manfredo Atzori %E Nikolay Burlutskiy %E Francesco Ciompi %E Zhang Li %E Fayyaz Minhas %E Henning Müller %E Tingying Peng %E Nasir Rajpoot %E Ben Torben-Nielsen %E Jeroen van der Laak %E Mitko Veta %E Yinyin Yuan %E Inti Zlobec %F pmlr-v156-marini21a %I PMLR %P 170--181 %U https://proceedings.mlr.press/v156/marini21a.html %V 156 %X Whole slide images (WSIs) are high-resolution digitized images of tissue samples, stored including different magnification levels. WSIs datasets often include only global annotations, available thanks to pathology reports. Global annotations refer to global findings in the high-resolution image and do not include information about the location of the regions of interest or the magnification levels used to identify a finding. This fact can limit the training of machine learning models, as WSIs are usually very large and each magnification level includes different information about the tissue. This paper presents a Multi-Scale Task Multiple Instance Learning (MuSTMIL) method, allowing to better exploit data paired with global labels and to combine contextual and detailed information identified at several magnification levels. The method is based on a multiple instance learning framework and on a multi-task network, that combines features from several magnification levels and produces multiple predictions (a global one and one for each magnification level involved). MuSTMIL is evaluated on colon cancer images, on binary and multilabel classification. MuSTMIL shows an improvement in performance in comparison to both single scale and another multi-scale multiple instance learning algorithm, demonstrating that MuSTMIL can help to better deal with global labels targeting full and multi-scale images.
APA
Marini, N., Otálora, S., Ciompi, F., Silvello, G., Marchesin, S., Vatrano, S., Buttafuoco, G., Atzori, M. & Müller, H.. (2021). Multi-Scale Task Multiple Instance Learning for the Classification of Digital Pathology Images with Global Annotations. Proceedings of the MICCAI Workshop on Computational Pathology, in Proceedings of Machine Learning Research 156:170-181 Available from https://proceedings.mlr.press/v156/marini21a.html.

Related Material