Continuous State-Space Models for Optimal Sepsis Treatment: a Deep Reinforcement Learning Approach

Aniruddh Raghu, Matthieu Komorowski, Leo Anthony Celi, Peter Szolovits, Marzyeh Ghassemi
Proceedings of the 2nd Machine Learning for Healthcare Conference, PMLR 68:147-163, 2017.

Abstract

Sepsis is a leading cause of mortality in intensive care units (ICUs) and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. Understanding more about a patient’s physiological state at a given time could hold the key to effective treatment policies. In this work, we propose a new approach to deduce optimal treatment policies for septic patients by using continuous state-space models and deep reinforcement learning. Learning treatment policies over continuous state-spaces is important, because doing so allows us to retain more of the patient’s physiological information. Our model is able to learn clinically interpretable treatment policies, similar in important aspects to the treatment policies of physicians. Evaluating our algorithm on past ICU patient data, we find that our model could reduce absolute patient mortality in the hospital by up to 3.6% over observed clinical policies. The learned treatment policies could be used to aid intensive care clinicians in medical decision making and improve the likelihood of patient survival.

Cite this Paper

BibTeX
@InProceedings{pmlr-v68-raghu17a, title = {Continuous State-Space Models for Optimal Sepsis Treatment: a Deep Reinforcement Learning Approach}, author = {Raghu, Aniruddh and Komorowski, Matthieu and Celi, Leo Anthony and Szolovits, Peter and Ghassemi, Marzyeh}, booktitle = {Proceedings of the 2nd Machine Learning for Healthcare Conference}, pages = {147--163}, year = {2017}, editor = {Doshi-Velez, Finale and Fackler, Jim and Kale, David and Ranganath, Rajesh and Wallace, Byron and Wiens, Jenna}, volume = {68}, series = {Proceedings of Machine Learning Research}, month = {18--19 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v68/raghu17a/raghu17a.pdf}, url = {https://proceedings.mlr.press/v68/raghu17a.html}, abstract = {Sepsis is a leading cause of mortality in intensive care units (ICUs) and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. Understanding more about a patient’s physiological state at a given time could hold the key to effective treatment policies. In this work, we propose a new approach to deduce optimal treatment policies for septic patients by using continuous state-space models and deep reinforcement learning. Learning treatment policies over continuous state-spaces is important, because doing so allows us to retain more of the patient’s physiological information. Our model is able to learn clinically interpretable treatment policies, similar in important aspects to the treatment policies of physicians. Evaluating our algorithm on past ICU patient data, we find that our model could reduce absolute patient mortality in the hospital by up to 3.6% over observed clinical policies. The learned treatment policies could be used to aid intensive care clinicians in medical decision making and improve the likelihood of patient survival.} }
Endnote
%0 Conference Paper %T Continuous State-Space Models for Optimal Sepsis Treatment: a Deep Reinforcement Learning Approach %A Aniruddh Raghu %A Matthieu Komorowski %A Leo Anthony Celi %A Peter Szolovits %A Marzyeh Ghassemi %B Proceedings of the 2nd Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2017 %E Finale Doshi-Velez %E Jim Fackler %E David Kale %E Rajesh Ranganath %E Byron Wallace %E Jenna Wiens %F pmlr-v68-raghu17a %I PMLR %P 147--163 %U https://proceedings.mlr.press/v68/raghu17a.html %V 68 %X Sepsis is a leading cause of mortality in intensive care units (ICUs) and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. Understanding more about a patient’s physiological state at a given time could hold the key to effective treatment policies. In this work, we propose a new approach to deduce optimal treatment policies for septic patients by using continuous state-space models and deep reinforcement learning. Learning treatment policies over continuous state-spaces is important, because doing so allows us to retain more of the patient’s physiological information. Our model is able to learn clinically interpretable treatment policies, similar in important aspects to the treatment policies of physicians. Evaluating our algorithm on past ICU patient data, we find that our model could reduce absolute patient mortality in the hospital by up to 3.6% over observed clinical policies. The learned treatment policies could be used to aid intensive care clinicians in medical decision making and improve the likelihood of patient survival.
APA
Raghu, A., Komorowski, M., Celi, L.A., Szolovits, P. & Ghassemi, M.. (2017). Continuous State-Space Models for Optimal Sepsis Treatment: a Deep Reinforcement Learning Approach. Proceedings of the 2nd Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 68:147-163 Available from https://proceedings.mlr.press/v68/raghu17a.html.

Related Material