Region-based Saliency Explanations on the Recognition of Facial Genetic Syndromes

Ömer Sümer, Rebekah L. Waikel, Suzanna E. Ledgister Hanchard, Dat Duong, Peter Krawitz, Cristina Conati, Benjamin D. Solomon, Elisabeth André
Proceedings of the 8th Machine Learning for Healthcare Conference, PMLR 219:712-736, 2023.

Abstract

Deep neural networks in computer vision have shown remarkable progress in recognizing facial genetic syndromes. Many genetic syndromes are difficult to detect, even for experienced clinicians, and computer-aided phenotyping can accelerate clinical diagnosis. High-stakes clinical tasks using deep learning, as in clinical genetics, require human understandable explanations for model decisions. Saliency methods are used to explain DNN predictions in various image analysis domains but have yet to be studied in facial genetics. The syndromic features of most genetic conditions are often localized to areas like the eyes, nose, and mouth. In this paper, to summarize the contribution of key facial regions to a specific disease prediction, we propose a face region relevance score that can be applied to any saliency method. We also investigate how prior knowledge, namely human phenotype ontology and DNN model explanations, align. Quantitative experiments are performed on a new database containing over 3,500 images of 11 rare facial syndromes, a healthy control group, and an additional test set of 171 facial images, whose respective facial phenotypes are labeled by clinicians. Current saliency methods are good at capturing dysmorphism in particular regions (parts of the face), but they may not completely capture all the relevant features in a given person or condition. Our study indicates which saliency explanations and face regions are more consistent with the phenotypes of specific genetic syndromes and could be used in large-scale clinical evaluations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v219-sumer23a, title = {Region-based Saliency Explanations on the Recognition of Facial Genetic Syndromes}, author = {S\"umer, \"Omer and Waikel, Rebekah L. and Hanchard, Suzanna E. Ledgister and Duong, Dat and Krawitz, Peter and Conati, Cristina and Solomon, Benjamin D. and Andr\'e, Elisabeth}, booktitle = {Proceedings of the 8th Machine Learning for Healthcare Conference}, pages = {712--736}, year = {2023}, editor = {Deshpande, Kaivalya and Fiterau, Madalina and Joshi, Shalmali and Lipton, Zachary and Ranganath, Rajesh and Urteaga, Iñigo and Yeung, Serene}, volume = {219}, series = {Proceedings of Machine Learning Research}, month = {11--12 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v219/sumer23a/sumer23a.pdf}, url = {https://proceedings.mlr.press/v219/sumer23a.html}, abstract = {Deep neural networks in computer vision have shown remarkable progress in recognizing facial genetic syndromes. Many genetic syndromes are difficult to detect, even for experienced clinicians, and computer-aided phenotyping can accelerate clinical diagnosis. High-stakes clinical tasks using deep learning, as in clinical genetics, require human understandable explanations for model decisions. Saliency methods are used to explain DNN predictions in various image analysis domains but have yet to be studied in facial genetics. The syndromic features of most genetic conditions are often localized to areas like the eyes, nose, and mouth. In this paper, to summarize the contribution of key facial regions to a specific disease prediction, we propose a face region relevance score that can be applied to any saliency method. We also investigate how prior knowledge, namely human phenotype ontology and DNN model explanations, align. Quantitative experiments are performed on a new database containing over 3,500 images of 11 rare facial syndromes, a healthy control group, and an additional test set of 171 facial images, whose respective facial phenotypes are labeled by clinicians. Current saliency methods are good at capturing dysmorphism in particular regions (parts of the face), but they may not completely capture all the relevant features in a given person or condition. Our study indicates which saliency explanations and face regions are more consistent with the phenotypes of specific genetic syndromes and could be used in large-scale clinical evaluations.} }
Endnote
%0 Conference Paper %T Region-based Saliency Explanations on the Recognition of Facial Genetic Syndromes %A Ömer Sümer %A Rebekah L. Waikel %A Suzanna E. Ledgister Hanchard %A Dat Duong %A Peter Krawitz %A Cristina Conati %A Benjamin D. Solomon %A Elisabeth André %B Proceedings of the 8th Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2023 %E Kaivalya Deshpande %E Madalina Fiterau %E Shalmali Joshi %E Zachary Lipton %E Rajesh Ranganath %E Iñigo Urteaga %E Serene Yeung %F pmlr-v219-sumer23a %I PMLR %P 712--736 %U https://proceedings.mlr.press/v219/sumer23a.html %V 219 %X Deep neural networks in computer vision have shown remarkable progress in recognizing facial genetic syndromes. Many genetic syndromes are difficult to detect, even for experienced clinicians, and computer-aided phenotyping can accelerate clinical diagnosis. High-stakes clinical tasks using deep learning, as in clinical genetics, require human understandable explanations for model decisions. Saliency methods are used to explain DNN predictions in various image analysis domains but have yet to be studied in facial genetics. The syndromic features of most genetic conditions are often localized to areas like the eyes, nose, and mouth. In this paper, to summarize the contribution of key facial regions to a specific disease prediction, we propose a face region relevance score that can be applied to any saliency method. We also investigate how prior knowledge, namely human phenotype ontology and DNN model explanations, align. Quantitative experiments are performed on a new database containing over 3,500 images of 11 rare facial syndromes, a healthy control group, and an additional test set of 171 facial images, whose respective facial phenotypes are labeled by clinicians. Current saliency methods are good at capturing dysmorphism in particular regions (parts of the face), but they may not completely capture all the relevant features in a given person or condition. Our study indicates which saliency explanations and face regions are more consistent with the phenotypes of specific genetic syndromes and could be used in large-scale clinical evaluations.
APA
Sümer, Ö., Waikel, R.L., Hanchard, S.E.L., Duong, D., Krawitz, P., Conati, C., Solomon, B.D. & André, E.. (2023). Region-based Saliency Explanations on the Recognition of Facial Genetic Syndromes. Proceedings of the 8th Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 219:712-736 Available from https://proceedings.mlr.press/v219/sumer23a.html.

Related Material