Reinforcement Learning For Sepsis Treatment: A Continuous Action Space Solution

Yong Huang, Rui Cao, Amir Rahmani
Proceedings of the 7th Machine Learning for Healthcare Conference, PMLR 182:631-647, 2022.

Abstract

Sepsis is the leading cause of death in intensive care units. It is challenging to treat sepsis because the optimal treatment is still unclear, and individual patients respond differently to treatments. Recent attempts to use reinforcement learning to provide real-time personalized treatment recommendations have shown promising results. However, the discrete action design (i.e., discretizing the continuum of action space into coarse-grained decisions) poses problems in policy learning and evaluation, and limits the effectiveness of the treatment recommendations. In this work, we proposed a continuous state and action space solution inspired by the Deep Deterministic Policy Gradient (DDPG) algorithm. We performed qualitative evaluations and applied the direct method for off-policy evaluations. Our results match clinician performance and are more clinically reasonable and explainable than the state of the art.

Cite this Paper

BibTeX
@InProceedings{pmlr-v182-huang22a, title = {Reinforcement Learning For Sepsis Treatment: A Continuous Action Space Solution}, author = {Huang, Yong and Cao, Rui and Rahmani, Amir}, booktitle = {Proceedings of the 7th Machine Learning for Healthcare Conference}, pages = {631--647}, year = {2022}, editor = {Lipton, Zachary and Ranganath, Rajesh and Sendak, Mark and Sjoding, Michael and Yeung, Serena}, volume = {182}, series = {Proceedings of Machine Learning Research}, month = {05--06 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v182/huang22a/huang22a.pdf}, url = {https://proceedings.mlr.press/v182/huang22a.html}, abstract = {Sepsis is the leading cause of death in intensive care units. It is challenging to treat sepsis because the optimal treatment is still unclear, and individual patients respond differently to treatments. Recent attempts to use reinforcement learning to provide real-time personalized treatment recommendations have shown promising results. However, the discrete action design (i.e., discretizing the continuum of action space into coarse-grained decisions) poses problems in policy learning and evaluation, and limits the effectiveness of the treatment recommendations. In this work, we proposed a continuous state and action space solution inspired by the Deep Deterministic Policy Gradient (DDPG) algorithm. We performed qualitative evaluations and applied the direct method for off-policy evaluations. Our results match clinician performance and are more clinically reasonable and explainable than the state of the art.} }
Endnote
%0 Conference Paper %T Reinforcement Learning For Sepsis Treatment: A Continuous Action Space Solution %A Yong Huang %A Rui Cao %A Amir Rahmani %B Proceedings of the 7th Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2022 %E Zachary Lipton %E Rajesh Ranganath %E Mark Sendak %E Michael Sjoding %E Serena Yeung %F pmlr-v182-huang22a %I PMLR %P 631--647 %U https://proceedings.mlr.press/v182/huang22a.html %V 182 %X Sepsis is the leading cause of death in intensive care units. It is challenging to treat sepsis because the optimal treatment is still unclear, and individual patients respond differently to treatments. Recent attempts to use reinforcement learning to provide real-time personalized treatment recommendations have shown promising results. However, the discrete action design (i.e., discretizing the continuum of action space into coarse-grained decisions) poses problems in policy learning and evaluation, and limits the effectiveness of the treatment recommendations. In this work, we proposed a continuous state and action space solution inspired by the Deep Deterministic Policy Gradient (DDPG) algorithm. We performed qualitative evaluations and applied the direct method for off-policy evaluations. Our results match clinician performance and are more clinically reasonable and explainable than the state of the art.
APA
Huang, Y., Cao, R. & Rahmani, A.. (2022). Reinforcement Learning For Sepsis Treatment: A Continuous Action Space Solution. Proceedings of the 7th Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 182:631-647 Available from https://proceedings.mlr.press/v182/huang22a.html.

Related Material