Improving the Calibration of Long Term Predictions of Heart Failure Rehospitalizations using Medical Concept Embedding

Sunil Kalmady, Weijie Sun, Justin Ezekowitz, Nowell Fine, Jonathan Howlett, Anamaria Savu, Russ Greiner, Padma Kaul
Proceedings of AAAI Spring Symposium on Survival Prediction - Algorithms, Challenges, and Applications 2021, PMLR 146:70-82, 2021.

Abstract

‘Medical concept embedding’ aims to provide vector representations of International Statistical Classification of Diseases (ICD) codes such that the relationship between two vectors mirrors the conceptual relationship between the two diagnoses or clinical interventions. Despite the growing interest in vector representations of clinical information in electronic health records (EHR), the utility of embedding methods has not been examined in the context of predicting individualized survival distributions (ISD). In this study, we apply ISD methods, specifically Cox-Proportional Hazards with Kalbfleisch-Prentice extension (CoxPH-KP) and Multi-task Logistic Regression (MTLR), to the task of predicting probability of Heart Failure (HF) rehospitalization or mortality, in a population-level database of 40,568 HF hospitalizations over the span of 8 years. Further, we compare performance of these ISD models with versus without code embeddings, that were learned in a temporally disjoint dataset of 229,359 all-cause hospitalizations. All our models show good discrimination in the validation dataset of 8,114 HF hospitalizations, with time-based concordance greater than 70% for every monthly intervals upto 8 years. Finally, we demonstrate that medical concept embedding does not always lead to improved model discrimination, but does improve model calibration, particularly over the longer time scales.

Cite this Paper

BibTeX
@InProceedings{pmlr-v146-kalmady21a, title = {Improving the Calibration of Long Term Predictions of Heart Failure Rehospitalizations using Medical Concept Embedding}, author = {Kalmady, Sunil and Sun, Weijie and Ezekowitz, Justin and Fine, Nowell and Howlett, Jonathan and Savu, Anamaria and Greiner, Russ and Kaul, Padma}, booktitle = {Proceedings of AAAI Spring Symposium on Survival Prediction - Algorithms, Challenges, and Applications 2021}, pages = {70--82}, year = {2021}, editor = {Greiner, Russell and Kumar, Neeraj and Gerds, Thomas Alexander and van der Schaar, Mihaela}, volume = {146}, series = {Proceedings of Machine Learning Research}, month = {22--24 Mar}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v146/kalmady21a/kalmady21a.pdf}, url = {https://proceedings.mlr.press/v146/kalmady21a.html}, abstract = {‘Medical concept embedding’ aims to provide vector representations of International Statistical Classification of Diseases (ICD) codes such that the relationship between two vectors mirrors the conceptual relationship between the two diagnoses or clinical interventions. Despite the growing interest in vector representations of clinical information in electronic health records (EHR), the utility of embedding methods has not been examined in the context of predicting individualized survival distributions (ISD). In this study, we apply ISD methods, specifically Cox-Proportional Hazards with Kalbfleisch-Prentice extension (CoxPH-KP) and Multi-task Logistic Regression (MTLR), to the task of predicting probability of Heart Failure (HF) rehospitalization or mortality, in a population-level database of 40,568 HF hospitalizations over the span of 8 years. Further, we compare performance of these ISD models with versus without code embeddings, that were learned in a temporally disjoint dataset of 229,359 all-cause hospitalizations. All our models show good discrimination in the validation dataset of 8,114 HF hospitalizations, with time-based concordance greater than 70% for every monthly intervals upto 8 years. Finally, we demonstrate that medical concept embedding does not always lead to improved model discrimination, but does improve model calibration, particularly over the longer time scales.} }
Endnote
%0 Conference Paper %T Improving the Calibration of Long Term Predictions of Heart Failure Rehospitalizations using Medical Concept Embedding %A Sunil Kalmady %A Weijie Sun %A Justin Ezekowitz %A Nowell Fine %A Jonathan Howlett %A Anamaria Savu %A Russ Greiner %A Padma Kaul %B Proceedings of AAAI Spring Symposium on Survival Prediction - Algorithms, Challenges, and Applications 2021 %C Proceedings of Machine Learning Research %D 2021 %E Russell Greiner %E Neeraj Kumar %E Thomas Alexander Gerds %E Mihaela van der Schaar %F pmlr-v146-kalmady21a %I PMLR %P 70--82 %U https://proceedings.mlr.press/v146/kalmady21a.html %V 146 %X ‘Medical concept embedding’ aims to provide vector representations of International Statistical Classification of Diseases (ICD) codes such that the relationship between two vectors mirrors the conceptual relationship between the two diagnoses or clinical interventions. Despite the growing interest in vector representations of clinical information in electronic health records (EHR), the utility of embedding methods has not been examined in the context of predicting individualized survival distributions (ISD). In this study, we apply ISD methods, specifically Cox-Proportional Hazards with Kalbfleisch-Prentice extension (CoxPH-KP) and Multi-task Logistic Regression (MTLR), to the task of predicting probability of Heart Failure (HF) rehospitalization or mortality, in a population-level database of 40,568 HF hospitalizations over the span of 8 years. Further, we compare performance of these ISD models with versus without code embeddings, that were learned in a temporally disjoint dataset of 229,359 all-cause hospitalizations. All our models show good discrimination in the validation dataset of 8,114 HF hospitalizations, with time-based concordance greater than 70% for every monthly intervals upto 8 years. Finally, we demonstrate that medical concept embedding does not always lead to improved model discrimination, but does improve model calibration, particularly over the longer time scales.
APA
Kalmady, S., Sun, W., Ezekowitz, J., Fine, N., Howlett, J., Savu, A., Greiner, R. & Kaul, P.. (2021). Improving the Calibration of Long Term Predictions of Heart Failure Rehospitalizations using Medical Concept Embedding. Proceedings of AAAI Spring Symposium on Survival Prediction - Algorithms, Challenges, and Applications 2021, in Proceedings of Machine Learning Research 146:70-82 Available from https://proceedings.mlr.press/v146/kalmady21a.html.

Related Material