EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins

Joshua Pickard, Wei Qi, Na Li, Ann Woolley, Lisa A. Cosimi, Roy Kishony, Deborah Hung
Proceedings of the 11th Machine Learning for Healthcare Conference, PMLR 340:1488-1516, 2026.

Abstract

Sepsis is a leading cause of mortality, yet optimal treatment policies remain contested. Existing reinforcement learning (RL) approaches learn fixed strategies for sepsis treatment, limiting adaptability to changing clinical objectives during inference. We propose EHR-MPC, a framework that decouples learning patient dynamics from optimizing treatment by training a patient digital twin in the form of a generative electronic health record (EHR) model. The digital twin predicts clinical trajectories under interventions and enables model predictive control (MPC) to optimize treatments via inference-time planning over simulations. We evaluate EHR-MPC on a multicenter ICU sepsis cohort spanning 8 hospitals in the Mass General Brigham health system using both off-policy importance sampling and on-policy simulation-based evaluation. Relative to RL baselines, EHR-MPC achieves comparable off-policy performance and improved simulation performance. Unlike RL, this work frames sepsis treatment optimization as inference-time control over learned patient dynamics, establishing a general framework for decision making with generative clinical models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v340-pickard26a, title = {EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins}, author = {Pickard, Joshua and Qi, Wei and Li, Na and Woolley, Ann and Cosimi, Lisa A. and Kishony, Roy and Hung, Deborah}, booktitle = {Proceedings of the 11th Machine Learning for Healthcare Conference}, pages = {1488--1516}, year = {2026}, editor = {Krishnan, Rahul G. and van Amsterdam, Wouter A. C. and Chopra, Sumit and Overgaard, Shauna and Hughes, Michael and Ötleş, Erkin and Shen, Yiqiu and Shanmugam, Divya and Nayan, Madhur and Engelhard, Matthew and Fackler, Jim and Oberst, Michael}, volume = {340}, series = {Proceedings of Machine Learning Research}, month = {12--14 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v340/main/assets/pickard26a/pickard26a.pdf}, url = {https://proceedings.mlr.press/v340/pickard26a.html}, abstract = {Sepsis is a leading cause of mortality, yet optimal treatment policies remain contested. Existing reinforcement learning (RL) approaches learn fixed strategies for sepsis treatment, limiting adaptability to changing clinical objectives during inference. We propose EHR-MPC, a framework that decouples learning patient dynamics from optimizing treatment by training a patient digital twin in the form of a generative electronic health record (EHR) model. The digital twin predicts clinical trajectories under interventions and enables model predictive control (MPC) to optimize treatments via inference-time planning over simulations. We evaluate EHR-MPC on a multicenter ICU sepsis cohort spanning 8 hospitals in the Mass General Brigham health system using both off-policy importance sampling and on-policy simulation-based evaluation. Relative to RL baselines, EHR-MPC achieves comparable off-policy performance and improved simulation performance. Unlike RL, this work frames sepsis treatment optimization as inference-time control over learned patient dynamics, establishing a general framework for decision making with generative clinical models.} }
Endnote
%0 Conference Paper %T EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins %A Joshua Pickard %A Wei Qi %A Na Li %A Ann Woolley %A Lisa A. Cosimi %A Roy Kishony %A Deborah Hung %B Proceedings of the 11th Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2026 %E Rahul G. Krishnan %E Wouter A. C. van Amsterdam %E Sumit Chopra %E Shauna Overgaard %E Michael Hughes %E Erkin Ötleş %E Yiqiu Shen %E Divya Shanmugam %E Madhur Nayan %E Matthew Engelhard %E Jim Fackler %E Michael Oberst %F pmlr-v340-pickard26a %I PMLR %P 1488--1516 %U https://proceedings.mlr.press/v340/pickard26a.html %V 340 %X Sepsis is a leading cause of mortality, yet optimal treatment policies remain contested. Existing reinforcement learning (RL) approaches learn fixed strategies for sepsis treatment, limiting adaptability to changing clinical objectives during inference. We propose EHR-MPC, a framework that decouples learning patient dynamics from optimizing treatment by training a patient digital twin in the form of a generative electronic health record (EHR) model. The digital twin predicts clinical trajectories under interventions and enables model predictive control (MPC) to optimize treatments via inference-time planning over simulations. We evaluate EHR-MPC on a multicenter ICU sepsis cohort spanning 8 hospitals in the Mass General Brigham health system using both off-policy importance sampling and on-policy simulation-based evaluation. Relative to RL baselines, EHR-MPC achieves comparable off-policy performance and improved simulation performance. Unlike RL, this work frames sepsis treatment optimization as inference-time control over learned patient dynamics, establishing a general framework for decision making with generative clinical models.
APA
Pickard, J., Qi, W., Li, N., Woolley, A., Cosimi, L.A., Kishony, R. & Hung, D.. (2026). EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins. Proceedings of the 11th Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 340:1488-1516 Available from https://proceedings.mlr.press/v340/pickard26a.html.

Related Material