Proceedings of Machine Learning Research

Addressing the Real-world Class Imbalance Problem in Dermatology

Mon, 23 Nov 2020 00:00:00 +0000

Class imbalance is a common problem in medical diagnosis, causing a standard classifier to be biased towards the common classes and perform poorly on the rare classes. This is especially true for dermatology, a specialty with thousands of skin conditions but many of which have low prevalence in the real world. Motivated by recent advances, we explore few-shot learning methods as well as conventional class imbalance techniques for the skin condition recognition problem and propose an evaluation setup to fairly assess the real-world utility of such approaches. We find the performance of few-show learning methods does not reach that of conventional class imbalance techniques, but combining the two approaches using a novel ensemble improves model performance, especially for rare classes. We conclude that ensembling can be useful to address the class imbalance problem, yet progress can further be accelerated by real-world evaluation setups for benchmarking new methods.

TL-Lite: Temporal Visualization and Learning for Clinical Forecasting

Mon, 23 Nov 2020 00:00:00 +0000

Clinical data extraction is a necessary step for quantitative analysis in clinical research. Whereas most machine learning algorithms learn from fixed length or regularly-collected panel data, health records data are neither. To facilitate the development of transparent and reproducible machine learning models from such data, we introduce TL-Lite, a clinical data ingestion, transformation, and visualization tool for conducting temporal machine learning. The central principle behind TL-Lite is to provide visual responsiveness at the individual level alongside management of the desired transformations behind the scenes that go on to be applied throughout the cohort and that result in cohort-level summaries, statistics, models and predictions. Characterization of the tool, discussion of design choices, and examples of use demonstrate its added value. A demo is provided at \url{https://www.andrew.cmu.edu/user/jweiss2/viz.html}.

Confounding Feature Acquisition for Causal Effect Estimation

Mon, 23 Nov 2020 00:00:00 +0000

Reliable treatment effect estimation from observational data depends on the availability of all confounding information. While much work has targeted treatment effect estimation from observational data, there is relatively little work in the setting of confounding variable missingness, where collecting more information on confounders is often costly or time-consuming. In this work, we frame this challenge as a problem of feature acquisition of confounding features for causal inference. Our goal is to prioritize acquiring values for a fixed and known subset of missing confounders in samples that lead to efficient average treatment effect estimation. We propose two acquisition strategies based on i) covariate balancing (CB), and ii) reducing statistical estimation error on observed factual outcome error (OE). We compare CB and OE on five common causal effect estimation methods, and demonstrate improved sample efficiency of OE over baseline methods under various settings. We also provide visualizations for further analysis on the difference between our proposed methods.

EEG-GCNN: Augmenting Electroencephalogram-based Neurological Disease Diagnosis using a Domain-guided Graph Convolutional Neural Network

Mon, 23 Nov 2020 00:00:00 +0000

This paper presents a novel graph convolutional neural network (GCNN)-based approach for improving the diagnosis of neurological diseases using scalp-electroencephalograms (EEGs). Although EEG is one of the main tests used for neurological-disease diagnosis, the sensitivity of EEG-based expert visual diagnosis remains at ∼50%. This indicates a clear need for advanced methodology to reduce the false negative rate in detecting abnormal scalp-EEGs. In that context, we focus on the problem of distinguishing the abnormal scalp EEGs of patients with neurological diseases, which were originally classified as ‘normal’ by experts, from the scalp EEGs of healthy individuals. The contributions of this paper are three-fold: 1) we present EEG-GCNN, a novel GCNN model for EEG data that captures both the spatial and functional connectivity between the scalp electrodes, 2) using EEG-GCNN, we perform the first large-scale evaluation of the aforementioned hypothesis, and 3) using two large scalp-EEG databases, we demonstrate that EEG-GCNN significantly outperforms the human baseline and classical machine learning (ML) baselines, with an AUC of 0.90.

Interpretable Epilepsy Detection in Routine, Interictal EEG Data using Deep Learning

Mon, 23 Nov 2020 00:00:00 +0000

Epilepsy, a common serious neurological disorder, is characterized by its frequently occurring seizures that cause its patients to be three times as likely to die prematurely. While the application of machine learning to EEG recordings has enabled the successful prediction of whether and when such seizures will occur, the reliable detection of epilepsy during seizure-free periods is lacking. As far as the authors are aware, this work proposes the first deep learning approach for the latter task – and the second machine learning approach altogether. Additionally, it does so in an interpretable fashion to validate the proposed method for a more wide-spread adoption in healthcare and to potentially unveil unknown epileptic biomarkers. The performance of the Tiny Visual Geometry Group (t-VGG) convolutional neural network is evaluated against Temple University Hospital’s \textit{EEG Epilepsy Corpus}, a data set of variable-length EEG recordings gathered during routine checkups. The t-VGG network predicted individual $10$ second EEG windows with an Area Under the Precision-Recall Curve (AUPR) of $93.02%$ for epileptic predictions and $55.85%$ for healthy ones – a significant improvement of respectively $7.24$pp and $18.6$pp ($p\!<\!.001$) over the current state-of-the-art. Averaging window predictions per recording improved the t-VGG’s respective AUPR performances further to $95.52%$ and $77.27%$. The Gradient-weighted Class Activation Mapping method for interpretability confirmed that the model was able to learn sensible features with connections to well-known epilepsy markers.

Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data

Mon, 23 Nov 2020 00:00:00 +0000

When deploying machine learning models in high-stakes real-world environments such as health care, it is crucial to accurately assess the uncertainty concerning a model’s prediction on abnormal inputs. However, there is a scarcity of literature analyzing this problem on medical data, especially on mixed-type tabular data such as Electronic Health Records. We close this gap by presenting a series of tests including a large variety of contemporary uncertainty estimation techniques, in order to determine whether they are able to identify out-ofdistribution (OOD) patients. In contrast to previous work, we design tests on realistic and clinically relevant OOD groups, and run experiments on real-world medical data. We find that almost all techniques fail to achieve convincing results, partly disagreeing with earlier findings.

Evaluation of Contrastive Predictive Coding for Histopathology Applications

Mon, 23 Nov 2020 00:00:00 +0000

Recent advances in self-supervised learning for image data are closing the gap between unsupervised and supervised learning. However, the effectiveness of self-supervised methods has primarily been demonstrated for natural images. If the results would extrapolate to histopathology images, there could be significant benefits due to the reduced need for annotated data. In this paper, Contrastive Predictive Coding (CPC), one of the most promising stateof-the-art self-supervised methods, is extensively evaluated on histology data by varying a range of different parameters, including training objective, resolution, and data setup. From the results, we are able to draw important conclusions on the usefulness of CPC for digital pathology. We show strong evidence of the limitations of the learned representation for tumor classification, where only low-level information learned early during training, in the first CPC layers, is used. Furthermore, in our experiments, diversifying the distribution of the dataset (i.e., data from multiple organs or medical centers) does not lead to the model learning a more general representation. This study deepens the understanding of how the CPC model’s objective relates to intrinsic characteristics of histology datasets and will help the development of effective self-supervised methods for histopathology.

Machine Learning for Health (ML4H) 2020: Advancing Healthcare for All

Mon, 23 Nov 2020 00:00:00 +0000

CheXphoto: 10,000+ Photos and Transformations of Chest X-rays for Benchmarking Deep Learning Robustness

Mon, 23 Nov 2020 00:00:00 +0000

Clinical deployment of deep learning algorithms for chest x-ray interpretation requires a solution that can integrate into the vast spectrum of clinical workflows across the world. An appealing approach to scaled deployment is to leverage the ubiquity of smartphones by capturing photos of x-rays to share with clinicians using messaging services like WhatsApp. However, the application of chest x-ray algorithms to photos of chest x-rays requires reliable classification in the presence of artifacts not typically encountered in digital x-rays used to train machine learning models. We introduce CheXphoto, a dataset of smartphone photos and synthetic photographic transformations of chest x-rays sampled from the CheXpert dataset. To generate CheXphoto we (1) automatically and manually captured photos of digital x-rays under different settings, and (2) generated synthetic transformations of digital x-rays targeted to make them look like photos of digital x-rays and x-ray films. We release this dataset as a resource for testing and improving the robustness of deep learning algorithms for automated chest x-ray interpretation on smartphone photos of chest x-rays.

ML4H Auditing: From Paper to Practice

Mon, 23 Nov 2020 00:00:00 +0000

Healthcare systems are currently adapting to digital technologies, producing large quantities of novel data. Based on these data, machine-learning algorithms have been developed to support practitioners in labor-intensive workflows such as diagnosis, prognosis, triage or treatment of disease. However, their translation into medical practice is often hampered by a lack of careful evaluation in different settings. Efforts have started worldwide to establish guidelines for evaluating machine learning for health (ML4H) tools, highlighting the necessity to evaluate models for bias, interpretability, robustness, and possible failure modes. However, testing and adopting these guidelines in practice remains an open challenge. In this work, we target the paper-to-practice gap by applying an ML4H audit framework proposed by the ITU/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) to three use cases: diagnostic prediction of diabetic retinopathy, diagnostic prediction of Alzheimer’s disease, and cytomorphologic classification for leukemia diagnostics. The assessment comprises dimensions such as bias, interpretability, and robustness. Our results highlight the importance of fine-grained and caseadapted quality assessment, provide support for incorporating proposed quality assessment considerations of ML4H during the entire development life cycle, and suggest improvements for future ML4H reference evaluation frameworks.

Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

Mon, 23 Nov 2020 00:00:00 +0000

Functional magnetic resonance imaging (fMRI) is a neuroimaging modality that captures the blood oxygen level in a subject’s brain while the subject either rests or performs a variety of functional tasks under different conditions. Given fMRI data, the problem of inferring the task, known as task state decoding, is challenging due to the high dimensionality (hundreds of million sampling points per datum) and complex spatio-temporal blood flow patterns inherent in the data. In this work, we propose to tackle the fMRI task state decoding problem by casting it as a 4D spatio-temporal classification problem. We present a novel architecture called Brain Attend and Decode (BAnD), that uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms for temporal modeling. We achieve significant performance gain compared to previous works on a 7-task benchmark from the large-scale Human Connectome Project-Young Adult (HCP-YA) dataset. We also investigate the transferability of BAnD’s extracted features on unseen HCP tasks, either by freezing the spatial feature extraction layers and retraining the temporal model, or finetuning the entire model. The pretrained features from BAnD are useful on similar tasks while finetuning them yields competitive results on unseen tasks/conditions.

A Neural SIR Model for Global Forecasting

Mon, 23 Nov 2020 00:00:00 +0000

Being able to understand and forecast epidemic developments is crucial for policymakers. We develop a predictive model combining epidemiological dynamics of compartmental models with highly non-linear interactions learned by a LSTM Network. A novel dynamic SIR model is fit to variables related to the population transmission of Covid-19. This is embedded in a Bayesian recursive updating framework which is then coupled with a LSTM network to forecast cases of Covid-19. The model significantly improves forecasts over simple univariate LSTM or SIR models. We apply the model to developed and developing countries and forecast confirmed infections and analyze future trajectories.

Contrastive Representation Learning for Electroencephalogram Classification

Mon, 23 Nov 2020 00:00:00 +0000

Interpreting and labeling human electroencephalogram (EEG) is a challenging task requiring years of medical training. We present a framework for learning representations from EEG signals via contrastive learning. By recombining channels from multi-channel recordings, we increase the number of samples quadratically per recording. We train a channel-wise feature extractor by extending the SimCLR framework to time-series data. We introduce a set of augmentations for EEG and study their efficacy on different classification tasks. We demonstrate that the learned features improve EEG classification and significantly reduce the amount of labeled data needed on three separate tasks: (1) Emotion Recognition (SEED), (2) Normal/Abnormal EEG classification (TUH), and (3) Sleep-stage scoring (SleepEDF). Our models show improved performance over previously reported supervised models on SEED and SleepEDF and self-supervised models on all three tasks.

3D Photography Based Neural Network Craniosynostosis Triaging System

Mon, 23 Nov 2020 00:00:00 +0000

Craniosynostosis (synostosis) is a serious disease where the sutures of a newborn’s skull fuse prematurely leading to debilitating head shape deformities. Due to the seriousness of this condition many normal infants and those with benign head shape abnormalities are referred to pediatric craniofacial plastic surgeons, leading to a high referral burden and delays in diagnosis for patients. A diagnostic delay beyond 4 months of age excludes patients from being treated with minimally invasive endoscopic procedures, leading to higher risk open surgeries. Machine learning (ML) image classifiers can enhance the triaging process of these referrals through the use of 3D images taken by a multi-camera & angle setup during patient visits. In doing so, children with synostosis can be identified earlier, qualifying them for less invasive endoscopic surgical intervention. After training a variety of convolutional neural network (CNN) models on 3D images supplemented with synthetic images using generative adversarial networks (GANs), the best-performing model was found to be a novel approach developed in our study called a multi-view collapsed 3D CNN, which achieved area under the receiver operating curves (AUROC) between 90.00-97.00% for detecting various sub-types of synostosis. These results demonstrate the ability for ML models to potentially streamline the detection of children with synostosis and help overcome challenges associated with high referral burdens for these patients.

Spectral discontinuity design: Interrupted time series with spectral mixture kernels

Mon, 23 Nov 2020 00:00:00 +0000

Quasi-experimental designs allow researchers to determine the effect of a treatment, even when randomized controlled trials are infeasible. A prominent example is interrupted time series (ITS) design, in which the effect of an intervention is determined by comparing the extrapolation of a model trained on data acquired up to moment of intervention, with the interpolation by a model trained on data up to the intervention. Typical approaches for ITS use (segmented) linear regression, and consequently ignore many of the spectral features of time series data. In this paper, we propose a Bayesian nonparametric approach to ITS, that uses Gaussian process regression and the spectral mixture kernel. This approach can capture more structure of the time series than traditional methods like linear regression or AR(I)MA models, which improves the extrapolation performance, and hence the accuracy of causal inference. We demonstrate our approach in simulations, and use it to determine the causal effect of Kundalini yoga meditation on heart rate oscillations. We show that our approach is able to detect the causal effect of interventions that alter the spectral features of these time series.

DeepHeartBeat: Latent trajectory learning of cardiac cycles using cardiac ultrasounds

Mon, 23 Nov 2020 00:00:00 +0000

Echocardiography monitors the heart movement for noninvasive diagnosis of heart diseases. It proves to be of profound practical importance as it combines low-cost portable instrumentation and rapid image acquisition without the risks of ionizing radiation. However, echocardiograms produce high-dimensional, noisy data which frequently proved difficult to interpret. As a solution, we propose a novel autoencoder-based framework, DeepHeartBeat, to learn human interpretable representations of cardiac cycles from cardiac ultrasound data. Our model encodes high dimensional observations by a cyclic trajectory in a lower dimensional space. We show that the learned parameters describing the latent trajectory are well interpretable and we demonstrate the versatility of our model by successfully applying it to various cardiologically relevant tasks, such as ejection fraction prediction and arrhythmia detection. As a result, DeepHeartBeat promises to serve as a valuable assistant tool for automating therapy decisions and guiding clinical care.

Appropriate Evaluation of Diagnostic Utility of Machine Learning Algorithm Generated Images

Mon, 23 Nov 2020 00:00:00 +0000

Generative machine learning (ML) methods can reduce time, cost, and radiation associated with medical image acquisition, compression, or generation techniques. While quantitative metrics are commonly used in the evaluation of ML generated images, it is unknown how well these quantitative metrics relate to the diagnostic utility of images. Here, fellowship-trained radiologists provided diagnoses and qualitative evaluations on chest radiographs reconstructed from the current standard JPEG2000 or variational autoencoder (VAE) techniques. Cohen’s kappa coefficient measured the agreement of diagnoses based on different reconstructions. Methods that produced similar Fréchet inception distance (FID) showed similar diagnostic performances. Thus in place of time-intensive expert radiologist verification, an appropriate target FID – an objective quantitative metric – can evaluate the clinical utility of ML generated medical images.

Improved Clinical Abbreviation Expansion via Non-Sense-Based Approaches

Mon, 23 Nov 2020 00:00:00 +0000

Abbreviation expansion is an important problem in clinical natural language processing because abbreviations often occur in text notes in medical records, and expansions of these abbreviations are critical for downstream applications such as assistive diagnosis and insurance code review. Previous studies have treated abbreviation expansion as a special case of word sense disambiguation; however, abbreviation expansion is easier because we only need the character level expansion and not necessarily the full sense of the abbreviation. In particular, such character level expansions may naturally occur elsewhere in medical contexts. Accordingly, we consider two categories of methods for abbreviation expansion: (a) non-sense-based methods that use information solely at lexical levels using state-of-the-art language models, and (b) sense-based methods that also incorporate sense information, such as glosses, from knowledge bases, to simultaneously perform the two tasks of expansion and disambiguation of the abbreviation. We propose two language model based approaches, including a novel length-agnostic permutation language model, find non-sense methods to be more effective than sense-based methods, and achieve the state-of-theart on three clinical datasets.

An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare

Mon, 23 Nov 2020 00:00:00 +0000

Reinforcement Learning (RL) has recently been applied to sequential estimation and prediction problems identifying and developing hypothetical treatment strategies for septic patients, with a particular focus on offline learning with observational data. In practice, successful RL relies on informative latent states derived from sequential observations to develop optimal treatment strategies. To date, how best to construct such states in a healthcare setting is an open question. In this paper, we perform an empirical study of several information encoding architectures using data from septic patients in the MIMIC-III dataset to form representations of a patient state. We evaluate the impact of representation dimension, correlations with established acuity scores, and the treatment policies derived from them. We find that sequentially formed state representations facilitate effective policy learning in batch settings, validating a more thoughtful approach to representation learning that remains faithful to the sequential and partial nature of healthcare data.

sEMG Gesture Recognition with a Simple Model of Attention

Mon, 23 Nov 2020 00:00:00 +0000

Myoelectric control is one of the leading areas of research in the field of robotic prosthetics. We present our research in surface electromyography (sEMG) signal classification, where our simple and novel attention-based approach now leads the industry, universally beating more complex, state-ofthe-art models. Our novel attention-based model achieves benchmark leading results on multiple industry-standard datasets including 53 finger, wrist, and grasping motions, improving over both sophisticated signal processing and CNN-based approaches. Our strong results with a straightforward model also indicate that sEMG represents a promising avenue for future machine learning research, with applications not only in prosthetics, but also in other important areas, such as diagnosis and prognostication of neurodegenerative diseases, computationally mediated surgeries, and advanced robotic control. We reinforce this suggestion with extensive ablative studies, demonstrating that a neural network can easily extract higher order spatiotemporal features from noisy sEMG data collected by affordable, consumer-grade sensors.

Parkinsonian Chinese Speech Analysis towards Automatic Classification of Parkinson's Disease

Mon, 23 Nov 2020 00:00:00 +0000

Speech disorders often occur at the early stage of Parkinson’s disease (PD). The speech impairments could be indicators of the disorder for early diagnosis, while motor symptoms are not obvious. In this study, we constructed a new speech corpus of Mandarin Chinese and addressed classification of patients with PD. We implemented classical machine learning methods with ranking algorithms for feature selection, convolutional and recurrent deep networks, and an end to end system. Our classification accuracy significantly surpassed state-of-the-art studies. The result suggests that free talk has stronger classification power than standard speech tasks, which could help the design of future speech tasks for efficient early diagnosis of the disease. Based on existing classification methods and our natural speech study, the automatic detection of PD from daily conversation could be accessible to the majority of the clinical population.

Neural Temporal Point Processes For Modelling Electronic Health Records

Mon, 23 Nov 2020 00:00:00 +0000

The modelling of Electronic Health Records (EHRs) has the potential to drive more efficient allocation of healthcare resources, enabling early intervention strategies and advancing personalised healthcare. However, EHRs are challenging to model due to their realisation as noisy, multi-modal data occurring at irregular time intervals. To address their temporal nature, we treat EHRs as samples generated by a Temporal Point Process (TPP), enabling us to model what happened in an event with when it happened in a principled way. We gather and propose neural network parameterisations of TPPs, collectively referred to as Neural TPPs. We perform evaluations on synthetic EHRs as well as on a set of established benchmarks. We show that TPPs significantly outperform their non-TPP counterparts on EHRs. We also show that an assumption of many Neural TPPs, that the class distribution is conditionally independent of time, reduces performance on EHRs. Finally, our proposed attention-based Neural TPP performs favourably compared to existing models, whilst aligning with real world interpretability requirements, an important step towards a component of clinical decision support systems.

A Bayesian Hierarchical Network for Combining Heterogeneous Data Sources in Medical Diagnoses

Mon, 23 Nov 2020 00:00:00 +0000

The increasingly widespread use of affordable, yet often less reliable medical data and diagnostic tools poses a new challenge for the field of ComputerAided Diagnosis: how can we combine multiple sources of information with varying levels of precision and uncertainty to provide an informative diagnosis estimate with confidence bounds? Motivated by a concrete application in lateral flow antibody testing, we devise a Stochastic Expectation-Maximization algorithm that allows the principled integration of heterogeneous and potentially unreliable data types. Our Bayesian formalism is essential in (a) flexibly combining these heterogeneous data sources and their corresponding levels of uncertainty, (b) quantifying the degree of confidence associated with a given diagnostic, and (c) dealing with the missing values that typically plague medical data. We quantify the potential of this approach on simulated data, and showcase its practicality by deploying it on a real COVID19 immunity study.

Quantifying Common Support between Multiple Treatment Groups Using a Contrastive-VAE

Mon, 23 Nov 2020 00:00:00 +0000

Estimating the effect of a given medical treatment on individual patients involves evaluating how clinical outcomes are affected by the treatment in question. Robust estimates of the treatment effect for a given patient with a prespecified set of clinical characteristics, are possible to obtain when there is sufficient common support for these features. Essentially, features having the greatest common support correspond to regions of significant overlap between the distributions of the different treatment groups. In observational datasets, however, all possible treatment options may not be uniformly represented, and therefore robust estimation of their effect may only be possible for the patients in the overlapping region. In this work, we propose a Contrastive Variational Autoencoder (ContrastiveVAE) to estimate where there is significant overlap between patient distributions corresponding to different treatment options. A Contrastive-VAE exploits shared information between different groups by modeling the shared information as arising from a shared set of latent variables to approximate distributions for treatment options that are not well represented in observational datasets. The result is an improved estimation of the distribution of the groups with a small number of data points. By estimating the likelihood for each group with annealed importance sampling, we are able to quantitatively identify the area of overlap between multiple treatment groups and obtain an effective confidence interval for the estimated individual treatment effect.

Zero-Shot Clinical Acronym Expansion via Latent Meaning Cells

Mon, 23 Nov 2020 00:00:00 +0000

We introduce Latent Meaning Cells, a deep latent variable model which learns contextualized representations of words by combining local lexical context and metadata. Metadata can refer to granular context, such as section type, or to more global context, such as unique document ids. Reliance on metadata for contextualized representation learning is apropos in the clinical domain where text is semi-structured and expresses high variation in topics. We evaluate the LMC model on the task of zero-shot clinical acronym expansion across three datasets. The LMC significantly outperforms a diverse set of baselines at a fraction of the pre-training cost and learns clinically coherent representations. We demonstrate that not only is metadata itself very helpful for the task, but that the LMC inference algorithm provides an additional large benefit.