Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series

Zachary C Lipton, David Kale, Randall Wetzel
Proceedings of the 1st Machine Learning for Healthcare Conference, PMLR 56:253-270, 2016.

Abstract

We demonstrate a simple strategy to cope with missing data in sequential inputs, addressing the task of multilabel classification of diagnoses given clinical time series. Collected from the intensive care unit (ICU) of a major urban medical center, our data consists of multivariate time series of observations. The data is irregularly sampled, leading to missingness patterns in re-sampled sequences. In this work, we show the remarkable ability of RNNs to make effective use of binary indicators to directly model missing data, improving AUC and F1significantly. However, while RNNs can learn arbitrary functions of the missing data and observations, linear models can only learn substitution values. For linear models and MLPs, we show an alternative strategy to capture this signal. Additionally, we evaluate LSTMs, MLPs, and linear models trained on missingness patterns only, showing that for several diseases, what tests are run can be more predictive than the results themselves.

Cite this Paper

BibTeX
@InProceedings{pmlr-v56-Lipton16, title = {Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series}, author = {Lipton, Zachary C and Kale, David and Wetzel, Randall}, booktitle = {Proceedings of the 1st Machine Learning for Healthcare Conference}, pages = {253--270}, year = {2016}, editor = {Doshi-Velez, Finale and Fackler, Jim and Kale, David and Wallace, Byron and Wiens, Jenna}, volume = {56}, series = {Proceedings of Machine Learning Research}, address = {Northeastern University, Boston, MA, USA}, month = {18--19 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v56/Lipton16.pdf}, url = {https://proceedings.mlr.press/v56/Lipton16.html}, abstract = {We demonstrate a simple strategy to cope with missing data in sequential inputs, addressing the task of multilabel classification of diagnoses given clinical time series. Collected from the intensive care unit (ICU) of a major urban medical center, our data consists of multivariate time series of observations. The data is irregularly sampled, leading to missingness patterns in re-sampled sequences. In this work, we show the remarkable ability of RNNs to make effective use of binary indicators to directly model missing data, improving AUC and F1significantly. However, while RNNs can learn arbitrary functions of the missing data and observations, linear models can only learn substitution values. For linear models and MLPs, we show an alternative strategy to capture this signal. Additionally, we evaluate LSTMs, MLPs, and linear models trained on missingness patterns only, showing that for several diseases, what tests are run can be more predictive than the results themselves.} }
Endnote
%0 Conference Paper %T Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series %A Zachary C Lipton %A David Kale %A Randall Wetzel %B Proceedings of the 1st Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2016 %E Finale Doshi-Velez %E Jim Fackler %E David Kale %E Byron Wallace %E Jenna Wiens %F pmlr-v56-Lipton16 %I PMLR %P 253--270 %U https://proceedings.mlr.press/v56/Lipton16.html %V 56 %X We demonstrate a simple strategy to cope with missing data in sequential inputs, addressing the task of multilabel classification of diagnoses given clinical time series. Collected from the intensive care unit (ICU) of a major urban medical center, our data consists of multivariate time series of observations. The data is irregularly sampled, leading to missingness patterns in re-sampled sequences. In this work, we show the remarkable ability of RNNs to make effective use of binary indicators to directly model missing data, improving AUC and F1significantly. However, while RNNs can learn arbitrary functions of the missing data and observations, linear models can only learn substitution values. For linear models and MLPs, we show an alternative strategy to capture this signal. Additionally, we evaluate LSTMs, MLPs, and linear models trained on missingness patterns only, showing that for several diseases, what tests are run can be more predictive than the results themselves.
RIS
TY - CPAPER TI - Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series AU - Zachary C Lipton AU - David Kale AU - Randall Wetzel BT - Proceedings of the 1st Machine Learning for Healthcare Conference DA - 2016/12/10 ED - Finale Doshi-Velez ED - Jim Fackler ED - David Kale ED - Byron Wallace ED - Jenna Wiens ID - pmlr-v56-Lipton16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 56 SP - 253 EP - 270 L1 - http://proceedings.mlr.press/v56/Lipton16.pdf UR - https://proceedings.mlr.press/v56/Lipton16.html AB - We demonstrate a simple strategy to cope with missing data in sequential inputs, addressing the task of multilabel classification of diagnoses given clinical time series. Collected from the intensive care unit (ICU) of a major urban medical center, our data consists of multivariate time series of observations. The data is irregularly sampled, leading to missingness patterns in re-sampled sequences. In this work, we show the remarkable ability of RNNs to make effective use of binary indicators to directly model missing data, improving AUC and F1significantly. However, while RNNs can learn arbitrary functions of the missing data and observations, linear models can only learn substitution values. For linear models and MLPs, we show an alternative strategy to capture this signal. Additionally, we evaluate LSTMs, MLPs, and linear models trained on missingness patterns only, showing that for several diseases, what tests are run can be more predictive than the results themselves. ER -
APA
Lipton, Z.C., Kale, D. & Wetzel, R.. (2016). Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series. Proceedings of the 1st Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 56:253-270 Available from https://proceedings.mlr.press/v56/Lipton16.html.

Related Material