- title: 'Conference on Health, Inference, and Learning (CHIL) 2022' volume: 174 URL: https://proceedings.mlr.press/v174/flores22a.html PDF: https://proceedings.mlr.press/v174/flores22a/flores22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-flores22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Ayah family: Zirikly - given: Michael C family: Hughes - given: Tasmie family: Sarker - given: Joyce C family: Ho - given: Tristan family: Naumann editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 1-4 id: flores22a issued: date-parts: - 2022 - 4 - 6 firstpage: 1 lastpage: 4 published: 2022-04-06 00:00:00 +0000 - title: 'Counterfactually Guided Policy Transfer in Clinical Settings' abstract: 'Domain shift, encountered when using a trained model for a new patient population, creates significant challenges for sequential decision making in healthcare since the target domain may be both data-scarce and confounded. In this paper, we propose a method for off-policy transfer by modeling the underlying generative process with a causal mechanism. We use informative priors from the source domain to augment counterfactual trajectories in the target in a principled manner. We demonstrate how this addresses data-scarcity in the presence of unobserved confounding. The causal parametrization of our sampling procedure guarantees that counterfactual quantities can be estimated from scarce observational target data, maintaining intuitive stability properties. Policy learning in the target domain is further regularized via the source policy through KL-divergence. Through evaluation on a simulated sepsis treatment task, our counterfactual policy transfer procedure significantly improves the performance of a learned treatment policy when assumptions of “no-unobserved confounding" are relaxed.' volume: 174 URL: https://proceedings.mlr.press/v174/killian22a.html PDF: https://proceedings.mlr.press/v174/killian22a/killian22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-killian22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Taylor W. family: Killian - given: Marzyeh family: Ghassemi - given: Shalmali family: Joshi editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 5-31 id: killian22a issued: date-parts: - 2022 - 4 - 6 firstpage: 5 lastpage: 31 published: 2022-04-06 00:00:00 +0000 - title: 'Evaluating Domain Generalization for Survival Analysis in Clinical Studies' abstract: 'Machine learning models are often required to generalize to new populations (domains) unseen during training, which may lead to model underperformance. So far, most research has focused on Domain Generalization methods for image classification tasks, which address the problem by learning domain invariant predictors. In this study, we assess the efficacy of domain generalization methods in survival analysis. The goal is to predict time-to-events such as death or disease progression based on baseline demographic and clinical variables of individuals exposed to medical treatment. We benchmark four domain generalization methods and several conventional/established methods on real world scenarios encountered in clinical practice. This includes tasks such as generalizing between randomized controlled trials to real world data, identification of prognostic models regardless of treatment or disease subtypes. We find that the generalization issue is often not as severe as reported in synthetic scenarios. Furthermore, our results corroborate previous findings that domain generalization often does not consistently outperform classical empirical risk minimization baselines also on low-dimensional data. Finally, to better understand when domain generalization methods can lead to performance gains and thus better outcomes for patients, we quantify the influence of different types of shifts occurring in the data.' volume: 174 URL: https://proceedings.mlr.press/v174/pfisterer22a.html PDF: https://proceedings.mlr.press/v174/pfisterer22a/pfisterer22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-pfisterer22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Florian family: Pfisterer - given: Chris family: Harbron - given: Gunther family: Jansen - given: Tao family: Xu editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 32-47 id: pfisterer22a issued: date-parts: - 2022 - 4 - 6 firstpage: 32 lastpage: 47 published: 2022-04-06 00:00:00 +0000 - title: 'Estimating Model Performance on External Samples from Their Limited Statistical Characteristics' abstract: 'Methods that address data shifts usually assume full access to multiple datasets. In the healthcare domain, however, privacy-preserving regulations as well as commercial interests limit data availability and, as a result, researchers can typically study only a small number of datasets. In contrast, limited statistical characteristics of specific patient samples are much easier to share and may be available from previously published literature or focused collaborative efforts. Here, we propose a method that estimates model performance in external samples from their limited statistical characteristics. We search for weights that induce internal statistics that are similar to the external ones; and that are closest to uniform. We then use model performance on the weighted internal sample as an estimation for the external counterpart. We evaluate the proposed algorithm on simulated data as well as electronic medical record data for two risk models, predicting complications in ulcerative colitis patients and stroke in women diagnosed with atrial fibrillation. In the vast majority of cases, the estimated external performance is much closer to the actual one than the internal performance. Our proposed method may be an important building block in training robust models and detecting potential model failures in external environments.' volume: 174 URL: https://proceedings.mlr.press/v174/el-hay22a.html PDF: https://proceedings.mlr.press/v174/el-hay22a/el-hay22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-el-hay22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Tal family: El-Hay - given: Chen family: Yanover editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 48-62 id: el-hay22a issued: date-parts: - 2022 - 4 - 6 firstpage: 48 lastpage: 62 published: 2022-04-06 00:00:00 +0000 - title: 'Enriching Unsupervised User Embedding via Medical Concepts' abstract: 'Clinical notes in Electronic Health Records (EHR) present rich documented information of patients to inference phenotype for disease diagnosis and study patient characteristics for cohort selection. Unsupervised user embedding aims to encode patients into fixed-length vectors without human supervisions. Medical concepts extracted from the clinical notes contain rich connections between patients and their clinical categories. However, existing \textit{unsupervised} approaches of user embeddings from clinical notes do not explicitly incorporate medical concepts. In this study, we propose a concept-aware unsupervised user embedding that jointly leverages text documents and medical concepts from two clinical corpora, MIMIC-III and Diabetes. We evaluate user embeddings on both extrinsic and intrinsic tasks, including phenotype classification, in-hospital mortality prediction, patient retrieval, and patient relatedness. Experiments on the two clinical corpora show our approach exceeds unsupervised baselines, and incorporating medical concepts can significantly improve the baseline performance.' volume: 174 URL: https://proceedings.mlr.press/v174/huang22a.html PDF: https://proceedings.mlr.press/v174/huang22a/huang22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-huang22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Xiaolei family: Huang - given: Franck family: Dernoncourt - given: Mark family: Dredze editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 63-78 id: huang22a issued: date-parts: - 2022 - 4 - 6 firstpage: 63 lastpage: 78 published: 2022-04-06 00:00:00 +0000 - title: 'Multi-Task Adversarial Learning for Treatment Effect Estimation in Basket Trials' abstract: 'Estimating treatment effects from observational data provides insights about causality guiding many real-world applications such as different clinical study designs, which are the formulations of trials, experiments, and observational studies in medical, clinical, and other types of research. In this paper, we describe causal inference for application in a novel clinical design called basket trial that tests how well a new drug works in patients who have different types of cancer that all have the same mutation. We propose a multi-task adversarial learning (MTAL) method, which incorporates feature selection multi-task representation learning and adversarial learning to estimate potential outcomes across different tumor types for patients sharing the same genetic mutation but having different tumor types. In our paper, the basket trial is employed as an intuitive example to present this new causal inference setting. This new causal inference setting includes, but is not limited to basket trials. This setting has the same challenges as the traditional causal inference problem, i.e., missing counterfactual outcomes under different subgroups and treatment selection bias due to confounders. We present the practical advantages of our MTAL method for the analysis of synthetic basket trial data and evaluate the proposed estimator on two benchmarks, IHDP and News. The results demonstrate the superiority of our MTAL method over the competing state-of-the-art methods.' volume: 174 URL: https://proceedings.mlr.press/v174/chu22a.html PDF: https://proceedings.mlr.press/v174/chu22a/chu22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-chu22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Zhixuan family: Chu - given: Stephen L family: Rathbun - given: Sheng family: Li editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 79-91 id: chu22a issued: date-parts: - 2022 - 4 - 6 firstpage: 79 lastpage: 91 published: 2022-04-06 00:00:00 +0000 - title: 'Neural Survival Clustering: Non-parametric mixture of neural networks for survival clustering' abstract: 'Survival analysis involves the modelling of the times to event. Proposed neural network approaches maximise the predictive performance of traditional survival models at the cost of their interpretability. This impairs their applicability in high stake domains such as medicine. Providing insights into the survival distributions would tackle this issue and advance the medical understanding of diseases. This paper approaches survival analysis as a mixture of neural baselines whereby different baseline cumulative hazard functions are modelled using positive and monotone neural networks. The efficiency of the solution is demonstrated on three datasets while enabling the discovery of new survival phenotypes.' volume: 174 URL: https://proceedings.mlr.press/v174/jeanselme22a.html PDF: https://proceedings.mlr.press/v174/jeanselme22a/jeanselme22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-jeanselme22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Vincent family: Jeanselme - given: Brian family: Tom - given: Jessica family: Barrett editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 92-102 id: jeanselme22a issued: date-parts: - 2022 - 4 - 6 firstpage: 92 lastpage: 102 published: 2022-04-06 00:00:00 +0000 - title: 'ADCB: An Alzheimer’s disease simulator for benchmarking observational estimators of causal effects' abstract: 'Simulators make unique benchmarks for causal effect estimation as they do not rely on unverifiable assumptions or the ability to intervene on real-world systems. This is especially important for estimators targeting healthcare applications as possibilities for experimentation are limited with good reason. We develop a simulator of clinical variables associated with Alzheimer’s disease, aimed to serve as a benchmark for causal effect estimation while modeling intricacies of healthcare data. We fit the system to the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset and ground hand-crafted components in results from comparative treatment trials and observational treatment patterns. The simulator includes parameters which alter the nature and difficulty of the causal inference tasks, such as latent variables, effect heterogeneity, length of observed subject history, behavior policy and sample size. We use the simulator to compare standard estimators of average and conditional treatment effects.' volume: 174 URL: https://proceedings.mlr.press/v174/kinyanjui22a.html PDF: https://proceedings.mlr.press/v174/kinyanjui22a/kinyanjui22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-kinyanjui22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Newton Mwai family: Kinyanjui - given: Fredrik D family: Johansson editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 103-118 id: kinyanjui22a issued: date-parts: - 2022 - 4 - 6 firstpage: 103 lastpage: 118 published: 2022-04-06 00:00:00 +0000 - title: 'Semi-Markov Offline Reinforcement Learning for Healthcare' abstract: 'Reinforcement learning (RL) tasks are typically framed as Markov Decision Processes (MDPs), assuming that decisions are made at fixed time intervals. However, many applications of great importance, including healthcare, do not satisfy this assumption, yet they are commonly modelled as MDPs after an artificial reshaping of the data. In addition, most healthcare (and similar) problems are \emph{offline} by nature, allowing for only retrospective studies. To address both challenges, we begin by discussing the Semi-MDP (SMDP) framework, which formally handles actions of variable timings. We next present a formal way to apply SMDP modifications to nearly any given value-based offline RL method. We use this theory to introduce three SMDP-based offline RL algorithms, namely, SDQN, SDDQN, and SBCQ. We then experimentally demonstrate that only these SMDP-based algorithms learn the optimal policy in variable-time environments, whereas their MDP counterparts do not. Finally, we apply our new algorithms to a real-world offline dataset pertaining to \emph{warfarin dosing for stroke prevention} and demonstrate similar results.' volume: 174 URL: https://proceedings.mlr.press/v174/fatemi22a.html PDF: https://proceedings.mlr.press/v174/fatemi22a/fatemi22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-fatemi22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Mehdi family: Fatemi - given: Mary family: Wu - given: Jeremy family: Petch - given: Walter family: Nelson - given: Stuart J family: Connolly - given: Alexander family: Benz - given: Anthony family: Carnicelli - given: Marzyeh family: Ghassemi editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 119-137 id: fatemi22a issued: date-parts: - 2022 - 4 - 6 firstpage: 119 lastpage: 137 published: 2022-04-06 00:00:00 +0000 - title: 'Uncertainty-Aware Text-to-Program for Question Answering on Structured Electronic Health Records' abstract: 'Question Answering on Electronic Health Records (EHR-QA) has a significant impact on the healthcare domain, and it is being actively studied. Previous research on structured EHR-QA focuses on converting natural language queries into query language such as SQL or SPARQL (NLQ2Query), so the problem scope is limited to pre-defined data types by the specific query language. In order to expand the EHR-QA task beyond this limitation to handle multi-modal medical data and solve complex inference in the future, more primitive systemic language is needed. In this paper, we design the program-based model (NLQ2Program) for EHR-QA as the first step towards the future direction. We tackle MIMICSPARQL*, the graph-based EHR-QA dataset, via a program-based approach in a semi-supervised manner in order to overcome the absence of gold programs. Without the gold program, our proposed model shows comparable performance to the previous state-of-the-art model, which is an NLQ2Query model (0.9% gain). In addition, for a reliable EHR-QA model, we apply the uncertainty decomposition method to measure the ambiguity in the input question. We empirically confirmed data uncertainty is most indicative of the ambiguity in the input question.' volume: 174 URL: https://proceedings.mlr.press/v174/kim22a.html PDF: https://proceedings.mlr.press/v174/kim22a/kim22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-kim22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Daeyoung family: Kim - given: Seongsu family: Bae - given: Seungho family: Kim - given: Edward family: Choi editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 138-151 id: kim22a issued: date-parts: - 2022 - 4 - 6 firstpage: 138 lastpage: 151 published: 2022-04-06 00:00:00 +0000 - title: 'Learning Unsupervised Representations for ICU Timeseries' abstract: 'Medical time series like physiological signals provide a rich source of information about patients’ underlying clinical states. Learning such states is a challenging problem for ML but has great utility for clinical applications. It allows us to identify patients with similar underlying conditions, track disease progression over time, and much more. The challenge with medical time series however, is the lack of well-defined labels for a given patient’s state for extended periods of time. Collecting such labels is expensive and often requires substantial effort. In this work, we propose an unsupervised representation learning method, called TRACE, that allows us to learn meaningful patient representations from time series collected in the Intensive Care Unit (ICU). We show the utility and generalizability of these representations in identifying different downstream clinical conditions and also show how the trajectory of representations over time exhibits progression toward critical conditions such as cardiopulmonary arrest or circulatory failure.' volume: 174 URL: https://proceedings.mlr.press/v174/weatherhead22a.html PDF: https://proceedings.mlr.press/v174/weatherhead22a/weatherhead22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-weatherhead22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Addison family: Weatherhead - given: Robert family: Greer - given: Michael-Alice family: Moga - given: Mjaye family: Mazwi - given: Danny family: Eytan - given: Anna family: Goldenberg - given: Sana family: Tonekaboni editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 152-168 id: weatherhead22a issued: date-parts: - 2022 - 4 - 6 firstpage: 152 lastpage: 168 published: 2022-04-06 00:00:00 +0000 - title: 'How to validate Machine Learning Models Prior to Deployment: Silent trial protocol for evaluation of real-time models at ICU' abstract: 'Rigorous evaluation of ML models prior to deployment in hospital settings is critical to ensure utility, performance, and safety. In addition, a guarantee of the usability of such tools requires careful user-centred design and evaluation. Such evaluations can be extra challenging for models that measure unquantified and complex clinical phenomena like the risk of deterioration. This paper introduces a silent trial protocol for evaluating models in real-time in the ICU setting. The trial is designed following principles of formative testing with the goal of evaluating model performance and gathering information that can be used to refine the model to best fit within the intended environment of deployment. We highlight the considerations for a systematic evaluation and explain the design and deployment of the components that enable this trial. We hope that the principles and considerations introduced in this paper can help other researchers validate ML models in their clinical settings.' volume: 174 URL: https://proceedings.mlr.press/v174/tonekaboni22a.html PDF: https://proceedings.mlr.press/v174/tonekaboni22a/tonekaboni22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-tonekaboni22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Sana family: Tonekaboni - given: Gabriela family: Morgenshtern - given: Azadeh family: Assadi - given: Aslesha family: Pokhrel - given: Xi family: Huang - given: Anand family: Jayarajan - given: Robert family: Greer - given: Gennady family: Pekhimenko - given: Melissa family: McCradden - given: Fanny family: Chevalier - given: Mjaye family: Mazwi - given: Anna family: Goldenberg editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 169-182 id: tonekaboni22a issued: date-parts: - 2022 - 4 - 6 firstpage: 169 lastpage: 182 published: 2022-04-06 00:00:00 +0000 - title: 'Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embedding' abstract: 'Increase in the use of Electronic Health Records (EHRs) has facilitated advances in predictive healthcare. However, EHR systems lack a unified code system for representing medical concepts. Heterogeneous formats of EHR present a barrier for the training and deployment of state-of-the-art deep learning models at scale. To overcome this problem, we introduce Description-based Embedding, DescEmb, a code-agnostic description-based representation learning framework for predictive modeling on EHR. DescEmb takes advantage of the flexibility of neural language models while maintaining a neutral approach that can be combined with prior frameworks for task-specific representation learning or predictive modeling. We test our model’s capacity on various experiments including prediction tasks, transfer learning and pooled learning. DescEmb shows higher performance in overall experiments compared to the code-based approach, opening the door to a text-based approach in predictive healthcare research that is not constrained by EHR structure nor special domain knowledge.' volume: 174 URL: https://proceedings.mlr.press/v174/hur22a.html PDF: https://proceedings.mlr.press/v174/hur22a/hur22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-hur22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Kyunghoon family: Hur - given: Jiyoung family: Lee - given: Jungwoo family: Oh - given: Wesley family: Price - given: Younghak family: Kim - given: Edward family: Choi editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 183-203 id: hur22a issued: date-parts: - 2022 - 4 - 6 firstpage: 183 lastpage: 203 published: 2022-04-06 00:00:00 +0000 - title: 'Improving the Fairness of Chest X-ray Classifiers' abstract: 'Deep learning models have reached or surpassed human-level performance in the field of medical imaging, especially in disease diagnosis using chest x-rays. However, prior work has found that such classifiers can exhibit biases in the form of gaps in predictive performance across protected groups. In this paper, we question whether striving to achieve zero disparities in predictive performance (i.e. group fairness) is the appropriate fairness definition in the clinical setting, over minimax fairness, which focuses on maximizing the performance of the worst-case group. We benchmark the performance of nine methods in improving classifier fairness across these two definitions. We find, consistent with prior work on non-clinical data, that methods which strive to achieve better worst-group performance do not outperform simple data balancing. We also find that methods which achieve group fairness do so by worsening performance for all groups. In light of these results, we discuss the utility of fairness definitions in the clinical setting, advocating for an investigation of the bias-inducing mechanisms in the underlying data distribution whenever possible.' volume: 174 URL: https://proceedings.mlr.press/v174/zhang22a.html PDF: https://proceedings.mlr.press/v174/zhang22a/zhang22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-zhang22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Haoran family: Zhang - given: Natalie family: Dullerud - given: Karsten family: Roth - given: Lauren family: Oakden-Rayner - given: Stephen family: Pfohl - given: Marzyeh family: Ghassemi editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 204-233 id: zhang22a issued: date-parts: - 2022 - 4 - 6 firstpage: 204 lastpage: 233 published: 2022-04-06 00:00:00 +0000 - title: 'Context-Sensitive Spelling Correction of Clinical Text via Conditional Independence' abstract: 'Spelling correction is a particularly important problem in clinical natural language processing because of the abundant occurrence of misspellings in medical records. However, the scarcity of labeled datasets in a clinical context makes it hard to build a machine learning system for such clinical spelling correction. In this work, we present a probabilistic model of correcting misspellings based on a simple conditional independence assumption, which leads to a modular decomposition into a language model and a corruption model. With a deep character-level language model trained on a large clinical corpus, and a simple edit-based corruption model, we can build a spelling correction model with small or no real data. Experimental results show that our model significantly outperforms baselines on two healthcare spelling correction datasets.' volume: 174 URL: https://proceedings.mlr.press/v174/kim22b.html PDF: https://proceedings.mlr.press/v174/kim22b/kim22b.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-kim22b.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Juyong family: Kim - given: Jeremy C family: Weiss - given: Pradeep family: Ravikumar editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 234-247 id: kim22b issued: date-parts: - 2022 - 4 - 6 firstpage: 234 lastpage: 247 published: 2022-04-06 00:00:00 +0000 - title: 'MedMCQA: A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering' abstract: 'This paper introduces MedMCQA, a new large-scale, Multiple-Choice Question Answering (MCQA) dataset designed to address real-world medical entrance exam questions. More than 194k high-quality AIIMS & NEET PG entrance exam MCQs covering 2.4k healthcare topics and 21 medical subjects are collected with an average token length of 12.77 and high topical diversity. Each sample contains a question, correct answer(s), and other options which requires a deeper language understanding as it tests the 10+ reasoning abilities of a model across a wide range of medical subjects & topics. A detailed explanation of the solution, along with the above information, is provided in this study.' volume: 174 URL: https://proceedings.mlr.press/v174/pal22a.html PDF: https://proceedings.mlr.press/v174/pal22a/pal22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-pal22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Ankit family: Pal - given: Logesh Kumar family: Umapathi - given: Malaikannan family: Sankarasubbu editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 248-260 id: pal22a issued: date-parts: - 2022 - 4 - 6 firstpage: 248 lastpage: 260 published: 2022-04-06 00:00:00 +0000 - title: 'Graph-Text Multi-Modal Pre-training for Medical Representation Learning' abstract: 'As the volume of Electronic Health Records (EHR) sharply grows, there has been emerging interest in learning the representation of EHR for healthcare applications. Representation learning of EHR requires appropriate modeling of the two dominant modalities in EHR: structured data and unstructured text. In this paper, we present MedGTX, a pre-trained model for multi-modal representation learning of the structured and textual EHR data. MedGTX uses a novel graph encoder to exploit the graphical nature of structured EHR data, and a text encoder to handle unstructured text, and a cross-modal encoder to learn a joint representation space. We pre-train our model through four proxy tasks on MIMIC-III, an open-source EHR data, and evaluate our model on two clinical benchmarks and three novel downstream tasks which tackle real-world problems in EHR data. The results consistently show the effectiveness of pre-training the model for joint representation of both structured and unstructured information from EHR. Given the promising performance of MedGTX, we believe this work opens a new door to jointly understanding the two fundamental modalities of EHR data.' volume: 174 URL: https://proceedings.mlr.press/v174/park22a.html PDF: https://proceedings.mlr.press/v174/park22a/park22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-park22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Sungjin family: Park - given: Seongsu family: Bae - given: Jiho family: Kim - given: Tackeun family: Kim - given: Edward family: Choi editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 261-281 id: park22a issued: date-parts: - 2022 - 4 - 6 firstpage: 261 lastpage: 281 published: 2022-04-06 00:00:00 +0000 - title: 'Data Augmentation for Electrocardiograms' abstract: 'Neural network models have demonstrated impressive performance in predicting pathologies and outcomes from the 12-lead electrocardiogram (ECG). However, these models often need to be trained with large, labelled datasets, which are not available for many predictive tasks of interest. In this work, we perform an empirical study examining whether training time data augmentation methods can be used to improve performance on such data-scarce ECG prediction problems. We investigate how data augmentation strategies impact model performance when detecting cardiac abnormalities from the ECG. Motivated by our finding that the effectiveness of existing augmentation strategies is highly task-dependent, we introduce a new method, \textit{TaskAug}, which defines a flexible augmentation policy that is optimized on a per-task basis. We outline an efficient learning algorithm to do so that leverages recent work in nested optimization and implicit differentiation. In experiments, considering three datasets and eight predictive tasks, we find that TaskAug is competitive with or improves on prior work, and the learned policies shed light on what transformations are most effective for different tasks. We distill key insights from our experimental evaluation, generating a set of best practices for applying data augmentation to ECG prediction problems.' volume: 174 URL: https://proceedings.mlr.press/v174/raghu22a.html PDF: https://proceedings.mlr.press/v174/raghu22a/raghu22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-raghu22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Aniruddh family: Raghu - given: Divya family: Shanmugam - given: Eugene family: Pomerantsev - given: John family: Guttag - given: Collin M family: Stultz editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 282-310 id: raghu22a issued: date-parts: - 2022 - 4 - 6 firstpage: 282 lastpage: 310 published: 2022-04-06 00:00:00 +0000 - title: 'Real-Time Seizure Detection using EEG: A Comprehensive Comparison of Recent Approaches under a Realistic Setting' abstract: 'Electroencephalogram (EEG) is an important diagnostic test that physicians use to record brain activity and detect seizures by monitoring the signals. There have been several attempts to detect seizures and abnormalities in EEG signals with modern deep learning models to reduce the clinical burden. However, they cannot be fairly compared against each other as they were tested in distinct experimental settings. Also, some of them are not trained in real-time seizure detection tasks, making it hard for on-device applications. In this work, for the first time, we extensively compare multiple state-of-the-art models and signal feature extractors in a real-time seizure detection framework suitable for real-world application, using various evaluation metrics including a new one we propose to evaluate more practical aspects of seizure detection models.' volume: 174 URL: https://proceedings.mlr.press/v174/lee22a.html PDF: https://proceedings.mlr.press/v174/lee22a/lee22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-lee22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Kwanhyung family: Lee - given: Hyewon family: Jeong - given: Seyun family: Kim - given: Donghwa family: Yang - given: Hoon-Chul family: Kang - given: Edward family: Choi editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 311-337 id: lee22a issued: date-parts: - 2022 - 4 - 6 firstpage: 311 lastpage: 337 published: 2022-04-06 00:00:00 +0000 - title: 'Lead-agnostic Self-supervised Learning for Local and Global Representations of Electrocardiogram' abstract: 'In recent years, self-supervised learning methods have shown significant improvement for pre-training with unlabeled data and have proven helpful for electrocardiogram signals. However, most previous pre-training methods for electrocardiogram focused on capturing only global contextual representations. This inhibits the models from learning fruitful representation of electrocardiogram, which results in poor performance on downstream tasks. Additionally, they cannot fine-tune the model with an arbitrary set of electrocardiogram leads unless the models were pre-trained on the same set of leads. In this work, we propose an ECG pre-training method that learns both local and global contextual representations for better generalizability and performance on downstream tasks. In addition, we propose random lead masking as an ECG-specific augmentation method to make our proposed model robust to an arbitrary set of leads. Experimental results on two downstream tasks, cardiac arrhythmia classification and patient identification, show that our proposed approach outperforms other state-of-the-art methods.' volume: 174 URL: https://proceedings.mlr.press/v174/oh22a.html PDF: https://proceedings.mlr.press/v174/oh22a/oh22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-oh22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Jungwoo family: Oh - given: Hyunseung family: Chung - given: Joon-myoung family: Kwon - given: Dong-gyun family: Hong - given: Edward family: Choi editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 338-353 id: oh22a issued: date-parts: - 2022 - 4 - 6 firstpage: 338 lastpage: 353 published: 2022-04-06 00:00:00 +0000 - title: 'PhysioMTL: Personalizing Physiological Patterns using Optimal Transport Multi-Task Regression' abstract: 'Heart rate variability (HRV) is a practical and noninvasive measure of autonomic nervous system activity, which plays an essential role in cardiovascular health. However, using HRV to assess physiology status is challenging. Even in clinical settings, HRV is sensitive to acute stressors such as physical activity, mental stress, hydration, alcohol, and sleep. Wearable devices provide convenient HRV measurements, but the irregularity of measurements and uncaptured stressors can bias conventional analytical methods. To better interpret HRV measurements for downstream healthcare applications, we learn a personalized diurnal rhythm as an accurate physiological indicator for each individual. We develop Physiological Multitask-Learning (PhysioMTL) by harnessing Optimal Transport theory within a Multitask-learning (MTL) framework. The proposed method learns an individual-specific predictive model from heterogeneous observations, and enables estimation of an optimal transport map that yields a push forward operation onto the demographic features for each task. Our model outperforms competing MTL methodologies on unobserved predictive tasks for synthetic and two real-world datasets. Specifically, our method provides remarkable prediction results on unseen held-out subjects given only $20%$ of the subjects in real-world observational studies. Furthermore, our model enables a counterfactual engine that generates the effect of acute stressors and chronic conditions on HRV rhythms.' volume: 174 URL: https://proceedings.mlr.press/v174/zhu22a.html PDF: https://proceedings.mlr.press/v174/zhu22a/zhu22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-zhu22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Jiacheng family: Zhu - given: Gregory family: Darnell - given: Agni family: Kumar - given: Ding family: Zhao - given: Bo family: Li - given: Xuanlong family: Nguyen - given: Shirley You family: Ren editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 354-374 id: zhu22a issued: date-parts: - 2022 - 4 - 6 firstpage: 354 lastpage: 374 published: 2022-04-06 00:00:00 +0000 - title: 'Disability prediction in multiple sclerosis using performance outcome measures and demographic data' abstract: 'Literature on machine learning for multiple sclerosis has primarily focused on the use of neuroimaging data such as magnetic resonance imaging and clinical laboratory tests for disease identification. However, studies have shown that these modalities are not consistent with disease activity such as symptoms or disease progression. Furthermore, the cost of collecting data from these modalities is high, leading to scarce evaluations. In this work, we used multi-dimensional, affordable, physical and smartphone-based performance outcome measures (POM) in conjunction with demographic data to predict multiple sclerosis disease progression. We performed a rigorous benchmarking exercise on two datasets and present results across 13 clinically actionable prediction endpoints and 6 machine learning models. To the best of our knowledge, our results are the first to show that it is possible to predict disease progression using POMs and demographic data in the context of both clinical trials and smartphone-based studies by using two datasets. Moreover, we investigate our models to understand the impact of different POMs and demographics on model performance through feature ablation studies. We also show that model performance is similar across different demographic subgroups (based on age and sex). To enable this work, we developed an end-to-end reusable pre-processing and machine learning framework which allows quicker experimentation over disparate MS datasets.' volume: 174 URL: https://proceedings.mlr.press/v174/roy22a.html PDF: https://proceedings.mlr.press/v174/roy22a/roy22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-roy22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Subhrajit family: Roy - given: Diana family: Mincu - given: Lev family: Proleev - given: Negar family: Rostamzadeh - given: Chintan family: Ghate - given: Natalie family: Harris - given: Christina family: Chen - given: Jessica family: Schrouff - given: Nenad family: Tomašev - given: Fletcher Lee family: Hartsell - given: Katherine family: Heller editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 375-396 id: roy22a issued: date-parts: - 2022 - 4 - 6 firstpage: 375 lastpage: 396 published: 2022-04-06 00:00:00 +0000 - title: 'Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation' abstract: 'Off-policy policy evaluation methods for sequential decision making can be used to help identify if a proposed decision policy is better than a current baseline policy. However, a new decision policy may be better than a baseline policy for some individuals but not others. This has motivated a push towards personalization and accurate per-state estimates of heterogeneous treatment effects (HTEs). Given the limited data present in many important applications such as health care, individual predictions can come at a cost to accuracy and confidence in such predictions. We develop a method to balance the need for personalization with confident predictions by identifying subgroups where it is possible to confidently estimate the expected difference in a new decision policy relative to a baseline. We propose a novel loss function that accounts for the uncertainty during the subgroup partitioning phase. In experiments, we show that our method can be used to form accurate predictions of HTEs where other methods struggle.' volume: 174 URL: https://proceedings.mlr.press/v174/keramati22a.html PDF: https://proceedings.mlr.press/v174/keramati22a/keramati22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-keramati22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Ramtin family: Keramati - given: Omer family: Gottesman - given: Leo Anthony family: Celi - given: Finale family: Doshi-Velez - given: Emma family: Brunskill editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 397-410 id: keramati22a issued: date-parts: - 2022 - 4 - 6 firstpage: 397 lastpage: 410 published: 2022-04-06 00:00:00 +0000 - title: 'Practical Challenges in Differentially-Private Federated Survival Analysis of Medical Data' abstract: 'Survival analysis or time-to-event analysis aims to model and predict the time it takes for an event of interest to happen in a population or an individual. In the medical context this event might be the time of dying, metastasis, recurrence of cancer, etc. Recently, the use of neural networks that are specifically designed for survival analysis has become more popular and an attractive alternative to more traditional methods. In this paper, we take advantage of the inherent properties of neural networks to federate the process of training of these models. This is crucial in the medical domain since data is scarce and collaboration of multiple health centers is essential to make a conclusive decision about the properties of a treatment or a disease. To ensure the privacy of the datasets, it is common to utilize differential privacy on top of federated learning. Differential privacy acts by introducing random noise to different stages of training, thus making it harder for an adversary to extract details about the data. However, in the realistic setting of small medical datasets and only a few data centers, this noise makes it harder for the models to converge. To address this problem, we propose DPFed-post which adds a post-processing stage to the private federated learning scheme. This extra step helps to regulate the magnitude of the noisy average parameter update and easier convergence of the model. For our experiments, we choose 3 real-world datasets in the realistic setting when each health center has only a few hundred records, and we show that DPFed-post successfully increases the performance of the models by an average of up to $17%$ compared to the standard differentially private federated learning scheme.' volume: 174 URL: https://proceedings.mlr.press/v174/rahimian22a.html PDF: https://proceedings.mlr.press/v174/rahimian22a/rahimian22a.pdf edit: https://github.com/mlresearch//v174/edit/gh-pages/_posts/2022-04-06-rahimian22a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the Conference on Health, Inference, and Learning' publisher: 'PMLR' author: - given: Shadi family: Rahimian - given: Raouf family: Kerkouche - given: Ina family: Kurth - given: Mario family: Fritz editor: - given: Gerardo family: Flores - given: George H family: Chen - given: Tom family: Pollard - given: Joyce C family: Ho - given: Tristan family: Naumann page: 411-425 id: rahimian22a issued: date-parts: - 2022 - 4 - 6 firstpage: 411 lastpage: 425 published: 2022-04-06 00:00:00 +0000