Predicting utilization of healthcare services from individual disease trajectories using RNNs with multi-headed attention

Yogesh Kumar, Henri Salo, Tuomo Nieminen, Kristian Vepsalainen, Sangita Kulathinal, Pekka Marttinen
Proceedings of the Machine Learning for Health NeurIPS Workshop, PMLR 116:93-111, 2020.

Abstract

Healthcare resource allocation is an application that has been largely neglected by the machine learning community. We utilize the Electronic Health Records (EHR) of 1.4 million Finnish citizens, aged 65 and above, to develop a sequential deep learning model to predict utilization of healthcare services in the following year on individual level. Historical longitudinal EHR records from previous years, consisting of diagnosis codes, procedures, and patient demographics, are used sequentially as an input to a Recurrent Neural Networks (RNN). We improve the standard RNN regression pipeline for EHR code sequences by adding a Convolutional Embedding layer to address multiple codes recorded simultaneously, and Multi-headed attention. This reduces the number of epochs to converge by approximately 38{%} while improving the accuracy. We achieve approximately 10{%} improvement in R 2 score compared with state-of-the-art count-based baselines. Finally, we demonstrate the model’s robustness to changes in healthcare practices over time, by showing that it retains it’s ability to predict well into future years without any data available at the time of prediction, which is needed in practice to aid the allocation of healthcare resources.

Cite this Paper

BibTeX
@InProceedings{pmlr-v116-kumar20a, title = {{Predicting utilization of healthcare services from individual disease trajectories using RNNs with multi-headed attention}}, author = {Kumar, Yogesh and Salo, Henri and Nieminen, Tuomo and Vepsalainen, Kristian and Kulathinal, Sangita and Marttinen, Pekka}, booktitle = {Proceedings of the Machine Learning for Health NeurIPS Workshop}, pages = {93--111}, year = {2020}, editor = {Dalca, Adrian V. and McDermott, Matthew B.A. and Alsentzer, Emily and Finlayson, Samuel G. and Oberst, Michael and Falck, Fabian and Beaulieu-Jones, Brett}, volume = {116}, series = {Proceedings of Machine Learning Research}, month = {13 Dec}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v116/kumar20a/kumar20a.pdf}, url = {https://proceedings.mlr.press/v116/kumar20a.html}, abstract = {Healthcare resource allocation is an application that has been largely neglected by the machine learning community. We utilize the Electronic Health Records (EHR) of 1.4 million Finnish citizens, aged 65 and above, to develop a sequential deep learning model to predict utilization of healthcare services in the following year on individual level. Historical longitudinal EHR records from previous years, consisting of diagnosis codes, procedures, and patient demographics, are used sequentially as an input to a Recurrent Neural Networks (RNN). We improve the standard RNN regression pipeline for EHR code sequences by adding a Convolutional Embedding layer to address multiple codes recorded simultaneously, and Multi-headed attention. This reduces the number of epochs to converge by approximately 38{%} while improving the accuracy. We achieve approximately 10{%} improvement in R 2 score compared with state-of-the-art count-based baselines. Finally, we demonstrate the model’s robustness to changes in healthcare practices over time, by showing that it retains it’s ability to predict well into future years without any data available at the time of prediction, which is needed in practice to aid the allocation of healthcare resources.} }
Endnote
%0 Conference Paper %T Predicting utilization of healthcare services from individual disease trajectories using RNNs with multi-headed attention %A Yogesh Kumar %A Henri Salo %A Tuomo Nieminen %A Kristian Vepsalainen %A Sangita Kulathinal %A Pekka Marttinen %B Proceedings of the Machine Learning for Health NeurIPS Workshop %C Proceedings of Machine Learning Research %D 2020 %E Adrian V. Dalca %E Matthew B.A. McDermott %E Emily Alsentzer %E Samuel G. Finlayson %E Michael Oberst %E Fabian Falck %E Brett Beaulieu-Jones %F pmlr-v116-kumar20a %I PMLR %P 93--111 %U https://proceedings.mlr.press/v116/kumar20a.html %V 116 %X Healthcare resource allocation is an application that has been largely neglected by the machine learning community. We utilize the Electronic Health Records (EHR) of 1.4 million Finnish citizens, aged 65 and above, to develop a sequential deep learning model to predict utilization of healthcare services in the following year on individual level. Historical longitudinal EHR records from previous years, consisting of diagnosis codes, procedures, and patient demographics, are used sequentially as an input to a Recurrent Neural Networks (RNN). We improve the standard RNN regression pipeline for EHR code sequences by adding a Convolutional Embedding layer to address multiple codes recorded simultaneously, and Multi-headed attention. This reduces the number of epochs to converge by approximately 38{%} while improving the accuracy. We achieve approximately 10{%} improvement in R 2 score compared with state-of-the-art count-based baselines. Finally, we demonstrate the model’s robustness to changes in healthcare practices over time, by showing that it retains it’s ability to predict well into future years without any data available at the time of prediction, which is needed in practice to aid the allocation of healthcare resources.
APA
Kumar, Y., Salo, H., Nieminen, T., Vepsalainen, K., Kulathinal, S. & Marttinen, P.. (2020). Predicting utilization of healthcare services from individual disease trajectories using RNNs with multi-headed attention. Proceedings of the Machine Learning for Health NeurIPS Workshop, in Proceedings of Machine Learning Research 116:93-111 Available from https://proceedings.mlr.press/v116/kumar20a.html.

Related Material