Explaining a machine learning decision to physicians via counterfactuals

Supriya Nagesh, Nina Mishra, Yonatan Naamad, James M Rehg, Mehul A Shah, Alexei Wagner
Proceedings of the Conference on Health, Inference, and Learning, PMLR 209:556-577, 2023.

Abstract

Machine learning models perform well on several healthcare tasks and can help reduce the burden on the healthcare system. However, the lack of explainability is a major roadblock to their adoption in hospitals. \textit{How can the decision of an ML model be explained to a physician?} The explanations considered in this paper are counterfactuals (CFs), hypothetical scenarios that would have resulted in the opposite outcome. Specifically, time-series CFs are investigated, inspired by the way physicians converse and reason out decisions ‘I would have given the patient a vasopressor if their blood pressure was lower and falling’. Key properties of CFs that are particularly meaningful in clinical settings are outlined: physiological plausibility, relevance to the task and sparse perturbations. Past work on CF generation does not satisfy these properties, specifically plausibility in that realistic time-series CFs are not generated. A variational autoencoder (VAE)-based approach is proposed that captures these desired properties. The method produces CFs that improve on prior approaches quantitatively (more plausible CFs as evaluated by their likelihood w.r.t original data distribution, and 100$\times$ faster at generating CFs) and qualitatively (2$\times$ more plausible and relevant) as evaluated by three physicians.

Cite this Paper

BibTeX
@InProceedings{pmlr-v209-nagesh23a, title = {Explaining a machine learning decision to physicians via counterfactuals}, author = {Nagesh, Supriya and Mishra, Nina and Naamad, Yonatan and Rehg, James M and Shah, Mehul A and Wagner, Alexei}, booktitle = {Proceedings of the Conference on Health, Inference, and Learning}, pages = {556--577}, year = {2023}, editor = {Mortazavi, Bobak J. and Sarker, Tasmie and Beam, Andrew and Ho, Joyce C.}, volume = {209}, series = {Proceedings of Machine Learning Research}, month = {22 Jun--24 Jun}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v209/nagesh23a/nagesh23a.pdf}, url = {https://proceedings.mlr.press/v209/nagesh23a.html}, abstract = {Machine learning models perform well on several healthcare tasks and can help reduce the burden on the healthcare system. However, the lack of explainability is a major roadblock to their adoption in hospitals. \textit{How can the decision of an ML model be explained to a physician?} The explanations considered in this paper are counterfactuals (CFs), hypothetical scenarios that would have resulted in the opposite outcome. Specifically, time-series CFs are investigated, inspired by the way physicians converse and reason out decisions ‘I would have given the patient a vasopressor if their blood pressure was lower and falling’. Key properties of CFs that are particularly meaningful in clinical settings are outlined: physiological plausibility, relevance to the task and sparse perturbations. Past work on CF generation does not satisfy these properties, specifically plausibility in that realistic time-series CFs are not generated. A variational autoencoder (VAE)-based approach is proposed that captures these desired properties. The method produces CFs that improve on prior approaches quantitatively (more plausible CFs as evaluated by their likelihood w.r.t original data distribution, and 100$\times$ faster at generating CFs) and qualitatively (2$\times$ more plausible and relevant) as evaluated by three physicians.} }
Endnote
%0 Conference Paper %T Explaining a machine learning decision to physicians via counterfactuals %A Supriya Nagesh %A Nina Mishra %A Yonatan Naamad %A James M Rehg %A Mehul A Shah %A Alexei Wagner %B Proceedings of the Conference on Health, Inference, and Learning %C Proceedings of Machine Learning Research %D 2023 %E Bobak J. Mortazavi %E Tasmie Sarker %E Andrew Beam %E Joyce C. Ho %F pmlr-v209-nagesh23a %I PMLR %P 556--577 %U https://proceedings.mlr.press/v209/nagesh23a.html %V 209 %X Machine learning models perform well on several healthcare tasks and can help reduce the burden on the healthcare system. However, the lack of explainability is a major roadblock to their adoption in hospitals. \textit{How can the decision of an ML model be explained to a physician?} The explanations considered in this paper are counterfactuals (CFs), hypothetical scenarios that would have resulted in the opposite outcome. Specifically, time-series CFs are investigated, inspired by the way physicians converse and reason out decisions ‘I would have given the patient a vasopressor if their blood pressure was lower and falling’. Key properties of CFs that are particularly meaningful in clinical settings are outlined: physiological plausibility, relevance to the task and sparse perturbations. Past work on CF generation does not satisfy these properties, specifically plausibility in that realistic time-series CFs are not generated. A variational autoencoder (VAE)-based approach is proposed that captures these desired properties. The method produces CFs that improve on prior approaches quantitatively (more plausible CFs as evaluated by their likelihood w.r.t original data distribution, and 100$\times$ faster at generating CFs) and qualitatively (2$\times$ more plausible and relevant) as evaluated by three physicians.
APA
Nagesh, S., Mishra, N., Naamad, Y., Rehg, J.M., Shah, M.A. & Wagner, A.. (2023). Explaining a machine learning decision to physicians via counterfactuals. Proceedings of the Conference on Health, Inference, and Learning, in Proceedings of Machine Learning Research 209:556-577 Available from https://proceedings.mlr.press/v209/nagesh23a.html.

Related Material