Dynamic Measurement Scheduling for Event Forecasting using Deep RL

Chun-Hao Chang, Mingjie Mai, Anna Goldenberg
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:951-960, 2019.

Abstract

Imagine a patient in critical condition. What and when should be measured to forecast detrimental events, especially under the budget constraints? We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed measurements. We learn our policy to be dynamically dependent on the patient’s health history. To scale our framework to exponentially large action space, we distribute our reward in a sequential setting that makes the learning easier. In our simulation, our policy outperforms heuristic-based scheduling with higher predictive gain and lower cost. In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by 31% or improve predictive gain by a factor of 3 as compared to physicians, under the off-policy policy evaluation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-chang19a, title = {Dynamic Measurement Scheduling for Event Forecasting using Deep {RL}}, author = {Chang, Chun-Hao and Mai, Mingjie and Goldenberg, Anna}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {951--960}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/chang19a/chang19a.pdf}, url = {https://proceedings.mlr.press/v97/chang19a.html}, abstract = {Imagine a patient in critical condition. What and when should be measured to forecast detrimental events, especially under the budget constraints? We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed measurements. We learn our policy to be dynamically dependent on the patient’s health history. To scale our framework to exponentially large action space, we distribute our reward in a sequential setting that makes the learning easier. In our simulation, our policy outperforms heuristic-based scheduling with higher predictive gain and lower cost. In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by 31% or improve predictive gain by a factor of 3 as compared to physicians, under the off-policy policy evaluation.} }
Endnote
%0 Conference Paper %T Dynamic Measurement Scheduling for Event Forecasting using Deep RL %A Chun-Hao Chang %A Mingjie Mai %A Anna Goldenberg %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-chang19a %I PMLR %P 951--960 %U https://proceedings.mlr.press/v97/chang19a.html %V 97 %X Imagine a patient in critical condition. What and when should be measured to forecast detrimental events, especially under the budget constraints? We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed measurements. We learn our policy to be dynamically dependent on the patient’s health history. To scale our framework to exponentially large action space, we distribute our reward in a sequential setting that makes the learning easier. In our simulation, our policy outperforms heuristic-based scheduling with higher predictive gain and lower cost. In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by 31% or improve predictive gain by a factor of 3 as compared to physicians, under the off-policy policy evaluation.
APA
Chang, C., Mai, M. & Goldenberg, A.. (2019). Dynamic Measurement Scheduling for Event Forecasting using Deep RL. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:951-960 Available from https://proceedings.mlr.press/v97/chang19a.html.

Related Material