Risk stratification through class-conditional conformal estimation: A strategy that improves the rule-out performance of MACE in the prehospital setting

Juan Jose Garcia, Nikhil Sarin, Rebecca R. Kitzmiller, Ashok Krishnamurthy, Jessica K. Zègre-Hemsey
Proceedings of the 9th Machine Learning for Healthcare Conference, PMLR 252, 2024.

Abstract

Accurate risk stratification of clinical scores is important to mitigate adverse outcomes in patient care. In this study we explore whether class-conditional conformal estimation can yield better risk stratification cutoffs, as measured by rule-out and rule-in performance. In the binary setting, the cutoffs are chosen to theoretically bound the false positive rate (FPR) and the false negative rate (FNR). We showcase rule-out performance improvements for the task of 30-day major adverse cardiac event (MACE) prediction in the prehospital setting over standard of care HEART and HEAR algorithms. Further, we observe the theoretical bounds materialize 96% and 77% of the time for FPR and FNR respectively across multiple datasets. Improving risk score accuracy is important since inaccurate stratification can lead to significant negative patient outcomes. For instance, in the case of MACE prediction, better rule-out performance translates into less delay of time dependent therapies that restore bloodflow to the compromised myocardium, thereby reducing morbidity and mortality.

Cite this Paper

BibTeX
@InProceedings{pmlr-v252-garcia24a, title = {Risk stratification through class-conditional conformal estimation: A strategy that improves the rule-out performance of {MACE} in the prehospital setting}, author = {Garcia, Juan Jose and Sarin, Nikhil and Kitzmiller, Rebecca R. and Krishnamurthy, Ashok and Z\`egre-Hemsey, Jessica K.}, booktitle = {Proceedings of the 9th Machine Learning for Healthcare Conference}, year = {2024}, editor = {Deshpande, Kaivalya and Fiterau, Madalina and Joshi, Shalmali and Lipton, Zachary and Ranganath, Rajesh and Urteaga, Iñigo}, volume = {252}, series = {Proceedings of Machine Learning Research}, month = {16--17 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v252/main/assets/garcia24a/garcia24a.pdf}, url = {https://proceedings.mlr.press/v252/garcia24a.html}, abstract = {Accurate risk stratification of clinical scores is important to mitigate adverse outcomes in patient care. In this study we explore whether class-conditional conformal estimation can yield better risk stratification cutoffs, as measured by rule-out and rule-in performance. In the binary setting, the cutoffs are chosen to theoretically bound the false positive rate (FPR) and the false negative rate (FNR). We showcase rule-out performance improvements for the task of 30-day major adverse cardiac event (MACE) prediction in the prehospital setting over standard of care HEART and HEAR algorithms. Further, we observe the theoretical bounds materialize 96% and 77% of the time for FPR and FNR respectively across multiple datasets. Improving risk score accuracy is important since inaccurate stratification can lead to significant negative patient outcomes. For instance, in the case of MACE prediction, better rule-out performance translates into less delay of time dependent therapies that restore bloodflow to the compromised myocardium, thereby reducing morbidity and mortality.} }
Endnote
%0 Conference Paper %T Risk stratification through class-conditional conformal estimation: A strategy that improves the rule-out performance of MACE in the prehospital setting %A Juan Jose Garcia %A Nikhil Sarin %A Rebecca R. Kitzmiller %A Ashok Krishnamurthy %A Jessica K. Zègre-Hemsey %B Proceedings of the 9th Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2024 %E Kaivalya Deshpande %E Madalina Fiterau %E Shalmali Joshi %E Zachary Lipton %E Rajesh Ranganath %E Iñigo Urteaga %F pmlr-v252-garcia24a %I PMLR %U https://proceedings.mlr.press/v252/garcia24a.html %V 252 %X Accurate risk stratification of clinical scores is important to mitigate adverse outcomes in patient care. In this study we explore whether class-conditional conformal estimation can yield better risk stratification cutoffs, as measured by rule-out and rule-in performance. In the binary setting, the cutoffs are chosen to theoretically bound the false positive rate (FPR) and the false negative rate (FNR). We showcase rule-out performance improvements for the task of 30-day major adverse cardiac event (MACE) prediction in the prehospital setting over standard of care HEART and HEAR algorithms. Further, we observe the theoretical bounds materialize 96% and 77% of the time for FPR and FNR respectively across multiple datasets. Improving risk score accuracy is important since inaccurate stratification can lead to significant negative patient outcomes. For instance, in the case of MACE prediction, better rule-out performance translates into less delay of time dependent therapies that restore bloodflow to the compromised myocardium, thereby reducing morbidity and mortality.
APA
Garcia, J.J., Sarin, N., Kitzmiller, R.R., Krishnamurthy, A. & Zègre-Hemsey, J.K.. (2024). Risk stratification through class-conditional conformal estimation: A strategy that improves the rule-out performance of MACE in the prehospital setting. Proceedings of the 9th Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 252 Available from https://proceedings.mlr.press/v252/garcia24a.html.

Related Material