Learning to Exploit Invariances in Clinical Time-Series Data using Sequence Transformer Networks

Jeeheh Oh, Jiaxuan Wang, Jenna Wiens
Proceedings of the 3rd Machine Learning for Healthcare Conference, PMLR 85:332-347, 2018.

Abstract

Recently, researchers have started applying convolutional neural networks (CNNs) with one-dimensional convolutions to clinical tasks involving time-series data. This is due, in part, to their computational efficiency, relative to recurrent neural networks and their ability to efficiently exploit certain temporal invariances, (\textit{e.g.}, phase invariance). However, it is well-established that clinical data may exhibit many other types of invariances (\textit{e.g.}, scaling). While preprocessing techniques, (\textit{e.g.,} dynamic time warping) may successfully transform and align inputs, their use often requires one to identify the types of invariances in advance. In contrast, we propose the use of Sequence Transformer Networks, an end-to-end trainable architecture that learns to identify and account for invariances in clinical time-series data. Applied to the task of predicting in-hospital mortality, our proposed approach achieves an improvement in the area under the receiver operating characteristic curve (AUROC) relative to a baseline CNN (AUROC=0.851 vs. AUROC=0.838). Our results suggest that a variety of valuable invariances can be learned directly from the data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v85-oh18a, title = {Learning to Exploit Invariances in Clinical Time-Series Data using Sequence Transformer Networks}, author = {Oh, Jeeheh and Wang, Jiaxuan and Wiens, Jenna}, booktitle = {Proceedings of the 3rd Machine Learning for Healthcare Conference}, pages = {332--347}, year = {2018}, editor = {Doshi-Velez, Finale and Fackler, Jim and Jung, Ken and Kale, David and Ranganath, Rajesh and Wallace, Byron and Wiens, Jenna}, volume = {85}, series = {Proceedings of Machine Learning Research}, month = {17--18 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v85/oh18a/oh18a.pdf}, url = {https://proceedings.mlr.press/v85/oh18a.html}, abstract = {Recently, researchers have started applying convolutional neural networks (CNNs) with one-dimensional convolutions to clinical tasks involving time-series data. This is due, in part, to their computational efficiency, relative to recurrent neural networks and their ability to efficiently exploit certain temporal invariances, (\textit{e.g.}, phase invariance). However, it is well-established that clinical data may exhibit many other types of invariances (\textit{e.g.}, scaling). While preprocessing techniques, (\textit{e.g.,} dynamic time warping) may successfully transform and align inputs, their use often requires one to identify the types of invariances in advance. In contrast, we propose the use of Sequence Transformer Networks, an end-to-end trainable architecture that learns to identify and account for invariances in clinical time-series data. Applied to the task of predicting in-hospital mortality, our proposed approach achieves an improvement in the area under the receiver operating characteristic curve (AUROC) relative to a baseline CNN (AUROC=0.851 vs. AUROC=0.838). Our results suggest that a variety of valuable invariances can be learned directly from the data.} }
Endnote
%0 Conference Paper %T Learning to Exploit Invariances in Clinical Time-Series Data using Sequence Transformer Networks %A Jeeheh Oh %A Jiaxuan Wang %A Jenna Wiens %B Proceedings of the 3rd Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2018 %E Finale Doshi-Velez %E Jim Fackler %E Ken Jung %E David Kale %E Rajesh Ranganath %E Byron Wallace %E Jenna Wiens %F pmlr-v85-oh18a %I PMLR %P 332--347 %U https://proceedings.mlr.press/v85/oh18a.html %V 85 %X Recently, researchers have started applying convolutional neural networks (CNNs) with one-dimensional convolutions to clinical tasks involving time-series data. This is due, in part, to their computational efficiency, relative to recurrent neural networks and their ability to efficiently exploit certain temporal invariances, (\textit{e.g.}, phase invariance). However, it is well-established that clinical data may exhibit many other types of invariances (\textit{e.g.}, scaling). While preprocessing techniques, (\textit{e.g.,} dynamic time warping) may successfully transform and align inputs, their use often requires one to identify the types of invariances in advance. In contrast, we propose the use of Sequence Transformer Networks, an end-to-end trainable architecture that learns to identify and account for invariances in clinical time-series data. Applied to the task of predicting in-hospital mortality, our proposed approach achieves an improvement in the area under the receiver operating characteristic curve (AUROC) relative to a baseline CNN (AUROC=0.851 vs. AUROC=0.838). Our results suggest that a variety of valuable invariances can be learned directly from the data.
APA
Oh, J., Wang, J. & Wiens, J.. (2018). Learning to Exploit Invariances in Clinical Time-Series Data using Sequence Transformer Networks. Proceedings of the 3rd Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 85:332-347 Available from https://proceedings.mlr.press/v85/oh18a.html.

Related Material