Proceedings of the 4th Machine Learning for Health Symposium Held in Vancouver, Canada on 15-16 December 2024 Published as Volume 259 by the Proceedings of Machine Learning Research on 17 February 2025. Volume Edited by: Stefan Hegselmann Helen Zhou Elizabeth Healey Trenton Chang Caleb Ellington Vishwali Mhasawade Sana Tonekaboni Peniel Argaw Haoran Zhang Series Editors: Neil D. Lawrence https://proceedings.mlr.press/v259/ Mon, 17 Feb 2025 09:08:41 +0000 Mon, 17 Feb 2025 09:08:41 +0000 Jekyll v3.10.0 Pregnancy is an intricate process involving substantial physiological changes that impact both maternal and fetal health. In this study, we demonstrate the ability to predict two common symptoms during the third trimester of pregnancy—edema and fatigue—using physiological measures from wearable devices and self-reported daily surveys from mobile apps. Our approach employs a Graph Neural Network (GNN) framework, enhanced with a modified weighted Cross-Entropy loss to improve prediction performance. The model achieved AUC scores of 77.27% for edema and 69.70% for fatigue. Additionally, we aligned data from self-reported pregnancy symptoms with clinical examinations to carefully select participant cohorts for our experiments. We also explored how various features identified by the GNN are linked to these symptoms, gaining deeper insights into the relationship between physiological measures and pregnancy symptoms. Our findings indicate that heart rate variability plays a significant role in predicting symptoms of edema and fatigue, and features related to low-intensity activity also have a notable impact. Some of our findings closely align with previous studies on pregnancy. Our research serves as a proof of concept that symptoms can be predicted using wearable data, which may enhance the immediate well-being of expectant mothers and potentially personalize the overall pregnancy experience. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/zhu25a.html https://proceedings.mlr.press/v259/zhu25a.html Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/zhou25a.html https://proceedings.mlr.press/v259/zhou25a.html Generating accurate radiology reports from medical images is a clinically important but challenging task. While current vision language models show promise, they are prone to generating hallucinations, potentially compromising patient care. We introduce RadFlag, a black-box method to enhance the accuracy of radiology report generation. Our method uses a sampling-based flagging technique to find hallucinatory generations that should be removed. We first sample multiple reports at varying temperatures and then use a large language model to identify claims that are not consistently supported across samples, indicating that the model has low confidence in those claims. Using a calibrated threshold, we flag a fraction of these claims as likely hallucinations, which should undergo extra review or be automatically rejected. Our method achieves high precision when identifying both individual hallucinatory sentences and reports that contain hallucinations. As an easy-to-use, black-box system that only requires access to a model’s temperature parameter, RadFlag is compatible with a wide range of radiology report generation models and has the potential to broadly improve the quality of automated radiology reporting. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/zhang25c.html https://proceedings.mlr.press/v259/zhang25c.html Healthcare applications require accurate and uncertainty-aware machine learning models, providing confidence rather than only blackbox predictions. However, training such deep learning models with insufficient data at individual sites (e.g., hospitals) poses a challenge. Federated learning (FL) mitigates this by allowing data holders to collaboratively train models without sharing sensitive health data. Yet, we identify two major realistic challenges when building uncertainty estimates in FL, severe data heterogeneity and high computational overhead. This paper proposes FedEE, an uncertainty-aware and efficient personalized FL framework for realistic healthcare applications. FedEE achieves an efficient way of ensembling by incorporating lightweight early exit blocks into a single backbone model. These blocks are personalized for each client to tackle data heterogeneity. Experiments with four FL strategies and three datasets demonstrate that FedEE achieves up to a 15% improvement in uncertainty estimation from vanilla softmax entropy and is competitive with expensive baselines, showcasing in the order of 5× improved efficiency with a 5-member ensemble. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/zhang25b.html https://proceedings.mlr.press/v259/zhang25b.html The high incidence and mortality rates associated with respiratory diseases underscores the importance of early screening. Machine learning models can automate clinical consultations and auscultation, offering vital support in this area. However, the data involved, spanning demographics, medical history, symptoms, and respiratory audio, are heterogeneous and complex. Existing approaches are insufficient and lack generalizability, as they typically rely on limited training data, basic fusion techniques, and task-specific design. In this paper, we propose RespLLM, a novel multimodal large language model (LLM) framework that unifies text and audio representations for respiratory health prediction. RespLLM leverages the extensive prior knowledge of pretrained LLMs and enables effective audio-text fusion through cross-modal attentions. Instruction tuning is employed to integrate diverse data from multiple sources, ensuring generalizability and versatility of the model. Experiments on five real-world datasets demonstrate that RespLLM outperforms leading baselines by an average of 4.6% on trained tasks, 7.9% on unseen datasets, and facilitates zero-shot predictions for new tasks. Our work lays the foundation for multimodal models that can perceive, listen to, and understand heterogeneous data, paving the way for scalable respiratory health diagnosis. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/zhang25a.html https://proceedings.mlr.press/v259/zhang25a.html The automated generation of free-text radiology reports is crucial for improving diagnosis and treatment in clinical practice. The latest chest X-ray report generation models utilize large vision language model (LVLM) architectures, which demand a higher level of interpretability for clinical deployment. Uncertainty estimation scores can assist clinicians in evaluating the reliability of these model outputs and promoting broader adoption of automated systems. In this paper, we conduct a comprehensive evaluation of the correlation between 16 LLM uncertainty scores and 6 radiology report evaluation metrics across 4 state-of-the-art LVLMs for CXR report generation. Our findings show a strong Pearson correlation, ranging from 0.4 to 0.6 on a scale from -1 to 1, for several models. We provide a detailed analysis of these uncertainty scores and evaluation metrics, offering insights in applying these methods in real clinical settings. This study is the first to evaluate LLM-based uncertainty estimation scores for X-ray report generation LVLM models, establishing a benchmark and laying the groundwork for their adoption in clinical practice. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/xu25a.html https://proceedings.mlr.press/v259/xu25a.html Automated medical coding, a clinical high-dimensional multilabel task, requires explicit interpretability. Existing works often rely on local interpretability methods, failing to provide comprehensive explanations of the overall mechanism behind each label prediction within a multilabel set. We propose a mechanistic interpretability module called DIctionary Label Attention (DILA) that disentangles uninterpretable dense embeddings into a sparse embedding space, where each nonzero element (a dictionary feature) represents a globally learned medical concept. Through human evaluations, we show that our sparse embeddings are more human understandable than its dense counterparts by at least 50 percent. Our automated dictionary feature identification pipeline, leveraging large language models (LLMs), uncovers thousands of learned medical concepts by examining and summarizing the highest activating tokens for each dictionary feature. We represent the relationships between dictionary features and medical codes through a sparse interpretable matrix, enhancing our global understanding of the model’s predictions while maintaining competitive performance and scalability without extensive human annotation. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/wu25a.html https://proceedings.mlr.press/v259/wu25a.html Precision medicine has the potential to tailor treatment decisions to individual patients using machine learning (ML) and artificial intelligence (AI), but it faces significant challenges due to complex biases in clinical observational data and the high-dimensional nature of biological data. This study models various types of treatment assignment biases using mutual information and investigates their impact on ML models for counterfactual prediction and biomarker identification. Unlike traditional counterfactual benchmarks that rely on fixed treatment policies, our work focuses on modeling different characteristics of the underlying observational treatment policy in distinct clinical settings. We validate our approach through experiments on toy datasets, semi-synthetic tumor cancer genome atlas (TCGA) data, and real-world biological outcomes from drug and CRISPR screens. By incorporating empirical biological mechanisms, we create a more realistic benchmark that reflects the complexities of real-world data. Our analysis reveals that different biases lead to varying model performances, with some biases, especially those unrelated to outcome mechanisms, having minimal effect on prediction accuracy. This highlights the crucial need to account for specific biases in clinical observational data in counterfactual ML model development, ultimately enhancing the personalization of treatment decisions in precision medicine. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/vollenweider25a.html https://proceedings.mlr.press/v259/vollenweider25a.html Survival analysis is a classic problem in statistics with important applications in healthcare. Most machine learning models for survival analysis are black-box models, limiting their use in healthcare settings where interpretability is paramount. More recently, glass-box machine learning models have been introduced for survival analysis, with both strong predictive performance and interpretability. Still, several gaps remain, as no prior glass-box survival model can produce calibrated shape functions with enough flexibility to capture the complex patterns often found in real data. To fill this gap, we introduce a new glass-box machine learning model for survival analysis called DNAMite. DNAMite uses feature discretization and kernel smoothing in its embedding module, making it possible to learn shape functions with a flexible balance of smoothness and jaggedness. Further, DNAMite produces calibrated shape functions that can be directly interpreted as contributions to the cumulative incidence function. Our experiments show that DNAMite generates shape functions closer to true shape functions on synthetic data, while making predictions with comparable predictive performance and better calibration than previous glass-box and black-box models Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/van-ness25a.html https://proceedings.mlr.press/v259/van-ness25a.html Imaging-based deep learning has transformed healthcare research, yet its clinical adoption remains limited due to challenges in comparing imaging models with traditional non-imaging and tabular data. To bridge this gap, we introduce Barttender, an interpretable framework that uses deep learning for the direct comparison of the utility of imaging versus non-imaging tabular data for tasks like disease prediction. Barttender converts non-imaging tabular features, such as scalar data from electronic health records, into grayscale bars, facilitating an interpretable and scalable deep learning based modeling of both data modalities. Our framework allows researchers to evaluate differences in utility through performance measures, as well as local (sample-level) and global (population-level) explanations. We introduce a novel measure to define global feature importances for image-based deep learning models, which we call gIoU. Experiments on the CheXpert and MIMIC datasets with chest X-rays and scalar data from electronic health records show that Barttender performs comparably to traditional methods and offers enhanced explainability using deep learning models. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/singla25a.html https://proceedings.mlr.press/v259/singla25a.html In current neuroimaging studies aimed at analysing BOLD signals, the focus has primarily been on correlation-based features derived from time series data. Considering Major Depressive Disorder (MDD), a widespread psychiatric condition, poses a complex and poorly understood pathology. Recent research has increasingly linked MDD to disruptions in brain connectivity, as observed through functional Magnetic Resonance Imaging (fMRI). Identifying the brain regions associated with neurological disorders and cognitive processes remains a central objective in neuroimaging studies. While Graph Neural Networks (GNNs) have been widely employed to extract disease-relevant information from fMRI data, existing methods face significant limitations. These limitations include neglecting the frequency-domain characteristics of neuronal interactions, inadequately incorporating non-imaging biomarkers such as sex and age, and paying insufficient attention to bias and model stability, which leaves models prone to small perturbations. We introduce CoRE-BOLD, a unified framework addressing these gaps for MDD diagnosis in this study. CoRE-BOLD employs an ensemble of stacked networks that learn complementary representations from both correlation- and coherence-based functional connectivities. To further improve the model, we enforce orthonormality constraints on the graph convolutional filters to enhance intra-network diversity and apply a diversity-maximizing regularizer for inter-network diversity. Unlike previous studies, which incorporate non-imaging sensitive attributes as biomarkers but inadvertently introduce bias, CoRE-BOLD mitigates this through a prejudice remover regularizer, promoting fairness in representation learning across both underrepresented and favored groups. Our experimental evaluation on the REST-meta-MDD dataset demonstrates the efficacy of CoRE-BOLD as a robust and fair framework for BOLD signal analysis in MDD detection, positioning it as a promising solution for real-world medical applications. Source code of CoRE-BOLD is freely available at: https://github.com/shashivipul/CoRE-BOLD.git. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/singh25a.html https://proceedings.mlr.press/v259/singh25a.html There is growing interest in applying AI to radiology report generation, particularly for chest X-rays (CXRs). This paper investigates whether incorporating pixel-level information through segmentation masks can improve fine-grained image interpretation of multimodal large language models (MLLMs) for radiology report generation. We introduce MAIRA-Seg, a segmentation-aware MLLM framework designed to utilize semantic segmentation masks alongside CXRs for generating radiology reports. We train expert segmentation models to obtain mask pseudolabels for radiology-specific structures in CXRs. Subsequently, building on the architectures of MAIRA, a CXR-specialised model for report generation, we integrate a trainable segmentation tokens extractor that leverages these mask pseudolabels, and employ mask-aware prompting to generate draft radiology reports. Our experiments on the publicly available MIMIC-CXR dataset show that MAIRA-Seg outperforms non-segmentation baselines. We also investigate set-of-marks prompting with MAIRA and find that MAIRA-Seg consistently demonstrates comparable or superior performance. The results confirm that using segmentation masks enhances the nuanced reasoning of MLLMs, potentially contributing to better clinical outcomes. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/sharma25a.html https://proceedings.mlr.press/v259/sharma25a.html Randomized control trials (RCTs) form a key tool for evaluating medical treatments. However commonly used techniques in RCT based treatment effect estimates suffer from two issues: a) ignoring spillover effects and b) focusing on average (or conditional) average effects. This is partly because evaluating counterfactual distributions is hard; and is further complicated by presence of interference. In this work we propose a new estimator, named RUMI, for estimating distributional quantities like CVaR, QTE from controlled trials under known interference. We provide theoretical justification behind our method and demonstrate its application using synthetic experiments and real data. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/shankar25a.html https://proceedings.mlr.press/v259/shankar25a.html Large language models (LLMs) have shown promise in safety-critical applications such as healthcare, yet the ability to quantify performance has lagged. An example of this challenge is in evaluating a summary of the patient’s medical record. A resulting summary can enable the provider to get a high-level overview of the patient’s health status quickly. Yet, a summary that omits important facts about the patient’s record can produce a misleading picture. This can lead to negative consequences on medical decision-making. We propose MED-OMIT as a metric to explore this challenge. We focus on using provider-patient history conversations to generate a subjective (a summary of the patient’s history) as a case study. We begin by discretizing facts from the dialogue and identifying which are omitted from the subjective. To determine which facts are clinically relevant, we measure the importance of each fact to a simulated differential diagnosis. We compare MED-OMIT’s performance to that of clinical experts and find broad agreement We use MED-OMIT to evaluate LLM performance on subjective generation and find some LLMs (gpt-4 and llama-3.1-405b) work well with little effort, while others (e.g. Llama 2) perform worse. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/schumacher25a.html https://proceedings.mlr.press/v259/schumacher25a.html Identifying patient eligibility for clinical trials is a critical bottleneck hindering medical research progress because many clinical trials allow only small, specific patient cohorts to be included and require a certain number of participating patients to yield definitive results. Manually screening patients through unstructured medical records is time-consuming and expensive. This paper explores the potential of large language models (LLMs) enhanced with medical context to automate patient eligibility assessments for clinical trials. We first design a two-stage zero-shot LLM approach to analyze a patient’s medical history (presented as unstructured text) and to determine their eligibility for a specific trial. We use advanced prompting strategies to guide the LLM toward faster and more targeted matches between trials and eligible patients. Additionally, a two-stage retrieval pipeline pre-filters potential trials using efficient retrieval techniques, reducing the number of trials considered for each patient. This two-way matching substantially improves processing speed and cost-effectiveness for clinical trial recruitment. Our method holds promise for streamlining clinical trial patient recruitment to accelerate medical advances. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/saeidi25a.html https://proceedings.mlr.press/v259/saeidi25a.html The risk of seizures in epilepsy fluctuates in cycles with multiday periodicity. The strength of these patient-specific seizure risk cycles can be modulated by disease processes. There is a lack of computational models of epilepsy that describe the progression and modulation of multiday seizure risk cycles. We developed a state space model (SSM) for epilepsy progression that learns individualized multiday seizure risk cycles from intracranial EEG (iEEG) data. To capture the cyclical nature of seizure risk, our model incorporated cyclical dynamics by using a special rotation matrix structure for the state transition matrix. The model learned patient-specific multiday cycles using a novel expectation-maximization algorithm. We evaluated the model on real-world data from one of the longest continuous iEEG recordings in people with epilepsy. The model forecast iEEG and inferred periods of heightened risk of seizures better than or comparable to baseline models, and provided novel insight into biological factors that modulate seizure risk cycles. To demonstrate the value of the model in developing brain stimulation treatment, the proposed SSM was integrated with reinforcement learning to reduce seizure risk in silico. Our model holds significant potential for addressing clinically important problems. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/saboo25a.html https://proceedings.mlr.press/v259/saboo25a.html Effective blood glucose forecasting is crucial for detecting events such as hypo- or hyperglycemia in people with diabetes, yet remains challenging in domains with only small, heterogeneous datasets, such as in the pediatric field. We present GluTFT, a novel transfer learning approach that allows leveraging models pretrained on publicly available adult diabetes datasets for pediatric glucose forecasting. We systematically evaluate multiple transfer learning strategies, including zero-shot prediction and fine-tuning across the entire dataset as well as specific subgroups of participants. Our extensive experiments reveal that GluTFT excels on the pretraining datasets and significantly outperforms baseline methods when fine-tuned. To validate the clinical relevance of our approach, we evaluate Parkes Error Grids, demonstrating the quality of GluTFT’s blood glucose forecasts and its potential for enhancing clinical decision-making for pediatric diabetes. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/ryser25a.html https://proceedings.mlr.press/v259/ryser25a.html Deep Learning (DL) has revolutionized medical image analysis by providing automated techniques to extract valuable insights from large datasets. However, challenges such as interpretability and reliance on extensive labeled data persist. Topological Data Analysis (TDA) has emerged as a complementary approach that captures underlying topological structures in data, potentially enhancing the performance of DL models. In this paper, we present a comprehensive evaluation of TDA methods for computer-aided diagnosis from two perspectives. First, we examine the effectiveness of topological methods in data-limited settings by comparing the standalone performance of DL models, TDA approaches, and their fusion. Our results demonstrate that integrating topological features into DL models significantly improves performance when labeled data are scarce. Second, we assess the standalone performance of TDA methods in data-rich environments using the MedMNIST collection, which includes over 600K images across 12 2D and 6 3D medical imaging datasets. Our experiments reveal that while TDA methods do not outperform DL models on 2D datasets, they achieve competitive results on 3D imaging tasks. These findings suggest that the fusion of TDA and DL methods can enhance the accuracy and robustness of computer-aided diagnosis, particularly in low-data or 3D imaging scenarios. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/nuwagira25a.html https://proceedings.mlr.press/v259/nuwagira25a.html This work introduces the first framework for reconstructing surgical dialogue from unstructured real-world recordings, which is crucial for characterizing teaching tasks. In surgical training, the formative verbal feedback that trainers provide to trainees during live surgeries is crucial for ensuring safety, correcting behavior immediately, and facilitating long-term skill acquisition. However, analyzing and quantifying this feedback is challenging due to its unstructured and specialized nature. Automated systems are essential to manage these complexities at scale, allowing for the creation of structured datasets that enhance feedback analysis and improve surgical education. Our framework integrates voice activity detection, speaker diarization, and automated speech recognition, with a novel enhancement that 1) removes hallucinations (non-existent utterances generated during speech recognition fueled by noise in the operating room) and 2) separates speech from trainers and trainees using few-shot voice samples. These aspects are vital for reconstructing accurate surgical dialogues and understanding the roles of operating room participants. Using data from 33 real-world surgeries, we demonstrated the system’s capability to reconstruct surgical teaching dialogues and detect feedback instances effectively (F1 score of 0.79±0.07). Moreover, our hallucination removal step improves feedback detection performance by ≈14%. Evaluation on downstream clinically relevant tasks of predicting Behavioral Adjustment of trainees and classifying Technical feedback, showed performances comparable to manual annotations with F1 scores of 0.82±0.03 and 0.81±0.03 respectively. These results highlight the effectiveness of our framework in supporting clinically relevant tasks and improving over manual methods. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/nasriddinov25a.html https://proceedings.mlr.press/v259/nasriddinov25a.html Interpreting and documenting findings from cardiac imaging studies is increasingly burdensome to readers in part due to the increasing amount of advanced cardiac imaging studies which capture multi-parametric data. This is particularly true of cardiac magnetic resonance imaging (CMR) studies which encode features of morphology, function, flow, parametric mapping, and myocardial viability in multiple 2D planes, but require a substantial amount of time to analyze, document, and integrate the numerous complex imaging features into a comprehensive report. Additionally, clearly communicating complex CMR findings and diagnoses to referring physicians with varying CMR knowledge and the ability to clinically correlated complex CMR findings is highly variable. Automatic interpretation and generation of the report have great potential to reduce the burden on readers and improve access through higher patient throughput. As such, there has been significant work in this area, although much of it has been focused on more simplistic chest X-ray and single view echocardiography. These data sources are represented by only a single view or have only a single source of contrast, greatly reducing the necessary complexity of the latent visual space. Furthermore, we recognize that clinical histories are important for accurate reporting. In this work, we propose to treat the CMR study as a multi-scene video and generate the corresponding report in an autoregressive manner. We further warm-start the generated report with the indications for the exam to improve the relevance of the generated report. We validate our model on two closed CMR datasets from two different institutions and demonstrate that our model offers significant improvements on both language generation metrics and human reader preference. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/nakashima25a.html https://proceedings.mlr.press/v259/nakashima25a.html The burden of injuries is essential to public health planning and policy making. Public health scientists rely on estimating the probability of nature-of-injuries (NI) for external causes of injuries (ECI) to calculate metrics used to describe burden of injuries globally. With more than 30 million records collected from 15 countries that include ECI with NI, in this study we develop a novel method to estimate probability of NI for ECI using self-supervised matrix imputation. We formulate learning the probability of NI for ECI for our data as a matrix imputation from noisy labels problem. Subsequently, we benchmark the collected data on 16 existing matrix imputation methods to uncover the best performing method for our data. Using self-supervision and data augmentation to curb the model’s tendency to overfit to noisy labels, our matrix imputation approach improves test set RMSE by 7.36% compared to the best performing imputation model used for benchmarking. In addition, the proposed self-supervised approach reduces the Euclidean distance of NI probabilities among age groups with similar probabilities by up to 20% without impacting model performance and uses counterfactual data augmentation (CDA) to mitigate potential biases from age, sex, platform, and country income status. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/naghavi25a.html https://proceedings.mlr.press/v259/naghavi25a.html Accurate diagnosis and prognosis assisted by pathology images are essential for cancer treatment selection and planning. Despite the recent trend of adopting deep-learning approaches for analyzing complex pathology images, they fall short as they often overlook the domain-expert understanding of tissue structure and cell composition. In this work, we focus on a challenging Open-ended Pathology VQA (PathVQA-Open) task and propose a novel framework named Path-RAG, which leverages HistoCartography to retrieve relevant domain knowledge from pathology images and significantly improves performance on PathVQA-Open. Admitting the complexity of pathology image analysis, Path-RAG adopts a human-centered AI approach by retrieving domain knowledge using HistoCartography to select the relevant patches from pathology images. Our experiments suggest that domain guidance can significantly boost the accuracy of LLaVA-Med from 38% to 47%, with a notable gain of 28% for H&E-stained pathology images in the PathVQA-Open dataset. For longer-form question and answer pairs, our model consistently achieves significant improvements of 32.5% in ARCH-Open PubMed and 30.6% in ARCH-Open Books on H&E images. All our relevant codes and datasets will be open-sourced. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/naeem25a.html https://proceedings.mlr.press/v259/naeem25a.html Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/mcdermott25a.html https://proceedings.mlr.press/v259/mcdermott25a.html Modeling policies for sequential clinical decision-making based on observational data is useful for describing treatment practices, standardizing frequent patterns in treatment, and evaluating alternative policies. For each task, it is essential that the policy model is interpretable. Learning accurate models requires effectively capturing a patient’s state, either through sequence representation learning or carefully crafted summaries of their medical history. While recent work has favored the former, it remains a question as to how histories should best be represented for interpretable policy modeling. Focused on model fit, we systematically compare diverse approaches to summarizing patient history for interpretable modeling of clinical policies across four sequential decision-making tasks. We illustrate differences in the policies learned using various representations by breaking down evaluations by patient subgroups, critical states, and stages of treatment, highlighting challenges specific to common use cases. We find that interpretable sequence models using learned representations perform on par with black-box models across all tasks. Interpretable models using hand-crafted representations perform substantially worse when ignoring history entirely, but are made competitive by incorporating only a few aggregated and recent elements of patient history. The added benefits of using a richer representation are pronounced for subgroups and in specific use cases. This underscores the importance of evaluating policy models in the context of their intended use. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/matsson25a.html https://proceedings.mlr.press/v259/matsson25a.html Matching donor organs to patients in need is a difficult but important problem. A crucial factor in transplant outcomes is the cold ischemic time of the organ, which increases every time an organ offer is refused. Despite this, acceptance dynamics have so far been neglected in favor of purely outcome driven offers. As a first alternative, we propose DynamITE, a novel organ allocation methodology that explicitly takes into account the acceptance behavior over sequences of offers. DynamITE dynamically updates organ acceptance estimates, cold ischemic times (CIT) and causal effects throughout the matching process. We demonstrate that DynamITE improves early organ acceptance and maximizes patient life expectancy compared to current policies. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/marchese25a.html https://proceedings.mlr.press/v259/marchese25a.html Generalist large language models (LLMs), not developed to do particular medical tasks, have achieved widespread use by the public. To avoid medical uses of these LLMs that have not been adequately tested and thus minimize any potential health risks, it is paramount that these models use adequate guardrails and safety measures. In this work, we propose a synthetic medical prompt generation method to evaluate generalist LLMs and enable red-teaming efforts. Using a commercial LLM and our dataset of synthetic user prompts, we illustrate how our methodology may used to identify responses for further evaluation and to assess whether guardrails are consistently implemented. Finally, we investigate the use of Flan-T5 in detecting LLM responses that offer unvetted medical advice and neglect to instruct users to consult with licensed professionals. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/lopez-martinez25a.html https://proceedings.mlr.press/v259/lopez-martinez25a.html Time series foundation models are pre-trained on large datasets and are able to achieve state-of-the-art performance in diverse tasks. However, to date, there has been limited work demonstrating how well these models perform in medical applications, where labeled data can be scarce. Further, we observe that currently, the majority of time series foundation models either are univariate in nature, or assume channel independence, meaning that they handle multivariate time series but do not model how the different variables relate. In this paper, we propose a prompt-tuning-inspired fine-tuning technique, Generalized Prompt Tuning (Gen-P-Tuning), that enables us to adapt an existing univariate time series foundation model (treated as frozen) to handle multivariate time series prediction. Our approach provides a way to combine information across channels (variables) of multivariate time series. We demonstrate the effectiveness of our fine-tuning approach against various baselines on two MIMIC classification tasks, and on influenza-like illness forecasting. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/liu25a.html https://proceedings.mlr.press/v259/liu25a.html Large language models with a transformer-based encoder/decoder architecture, such as T5, have become standard platforms for supervised tasks. To bring these technologies to the clinical domain, recent work has trained new or adapted existing models to clinical data. However, the evaluation of these clinical T5 models and comparison to other models has been limited. Are the clinical T5 models better choices than FLAN-tuned generic T5 models? Do they generalize better to new clinical domains that differ from the training sets? We comprehensively evaluate these models across several clinical tasks and domains. We find that clinical T5 models provide marginal improvements over existing models, and perform worse when evaluated on different domains. Our results inform future choices in developing clinical LLMs. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/li25a.html https://proceedings.mlr.press/v259/li25a.html Federated learning (FL) is a promising paradigm to collaboratively train a global chest x-ray (CXR) classification model using distributed datasets while preserving patient privacy. A significant, yet relatively underexplored, challenge in FL is class-heterogeneity, where clients have different sets of classes. We propose surgical aggregation, a FL method that uses selective aggregation to collaboratively train a global model using distributed, class-heterogeneous datasets. Unlike other methods, our method does not rely on the assumption that clients share the same classes as other clients, know the classes of other clients, or have access to a fully annotated dataset. We evaluate surgical aggregation using class-heterogeneous CXR datasets across IID and non-IID settings. Our results show that our method outperforms current methods and has better generalizability. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/kulkarni25a.html https://proceedings.mlr.press/v259/kulkarni25a.html Intimate Partner Violence (IPV) among adolescents is a major public health concern, particularly for justice-involved adolescents which are at higher risk of experiencing IPV. Early detection of disagreements between romantic partners can provide an opportunity for just-in-time interventions to prevent escalation. Prior work has proposed methods for early detection of disagreements based on metadata features of text message conversations. In this work, we build on these prior efforts and investigate the impact of explicitly modeling the contents of text conversations for disagreement detection. We develop and evaluate supervised classifiers that combine metadata features with sentiment and semantic features of texts and compare their performance against few-shot learning with instruction-tuned Large Language Models (LLMs). We conduct experiments on a dataset collected to study the communication patterns and risk factors associated with IPV among justice-involved adolescents. In addition, we measure models’ generalization to out-of-distribution samples using an external dataset comprising adolescents enrolled in child welfare services. We find that: (i) text-based features improve predictive performance but do not help models generalize to other populations; and (ii) LLMs struggle in this setting but can outperform supervised classifiers in out-of-distribution samples. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/kommalapati25a.html https://proceedings.mlr.press/v259/kommalapati25a.html Alignment of AI models to ensure that they are safe and useful decision-aids or decision-makers in human society is close to the top of the technical concerns of many if not most major AI deployment efforts. Here I explore a class of categorical decisions, triage, for which AI models are already being used in medicine. I use this to motivate the urgent need for an international Human Values Project (HVP) that will be at least as hungry for empirical data as any existing international project. A major component of the HVP will be the Clinical Decision Dynamics Study that captures descriptive and normative decisions across a wide range of clinical context and from highly diverse, lay and professional perspectives. Along the way, there are important preliminary studies that the AI (and medicine) community should embrace. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/kohane25a.html https://proceedings.mlr.press/v259/kohane25a.html Cardiac MRI is a powerful diagnostic tool, but traditional analysis relies on complex segmentation-based workflows that may provide only a partial picture of cardiovascular health. To address these limitations, we introduce HeartMAE, a novel framework that uses masked autoencoding (MAE) to learn features directly from cardiac MRIs, without any labels. By incorporating optical flow during training, HeartMAE is guided towards cardiac regions, which significantly improves its downstream performance. A multitask model, built on a shared HeartMAE embedding layer, accurately predicts key cardiac health indicators, extracardiac features and major cardiovascular conditions. Moreover, HeartMAE features may be used as embeddings for clustering to enable patient stratification. Requiring only MRI data, HeartMAE is highly scalable and adaptable to larger datasets, paving the way for foundation models in cardiac imaging. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/kim25a.html https://proceedings.mlr.press/v259/kim25a.html Meta-analyses are usually conducted on small amounts of “trusted” data, ideally from randomized, controlled trials. Excluding untrusted (observational) data — such as medical records and related scientific literature — avoids potential confounding and ensures unbiased conclusions. Unfortunately, this exclusion can reduce predictive accuracy to the point of clinical irrelevance, especially when trials are heterogeneous. This paper shows how untrusted data can be safely incorporated into meta-analysis, improving predictions without sacrificing rigor or introducing unproven assumptions. Our approach, called conformal meta-analysis, consists of (1) learning a (potentially flawed) prior distribution from the untrusted data, (2) using the prior and trusted data to derive a simple, fully-conformal prediction interval for the observed trial effect, and (3) analytically extracting an interval for the true (unobserved) effect. In multiple experiments on healthcare datasets, our algorithms deliver tighter, sounder intervals than traditional ones. This paper conceptually realigns meta-analysis as a foundation for evidence-based medicine, embracing heterogeneity and untrusted data for more nuanced, precise predictions. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/kaul25a.html https://proceedings.mlr.press/v259/kaul25a.html Remote physiological monitoring presents an opportunity to enhance patient care, particularly in scenarios where traditional monitoring methods are impractical or unavailable. Heart rate, being a principal indicator of health, has been a focal point of video-based monitoring systems. Despite significant advancements in remote photoplethysmography technology, several challenges persist, including motion artifacts, data homogeneity and availability, which impact the accuracy and reliability of such solutions. In this work, we introduce a novel framework aimed at addressing these challenges, ST2S-rPPG. Our methodology involves a stabilization method to mitigate motion artifacts. We propose a spatiotemporal representation of video data, which captures predictive available information in the video and assists in transforming the input video. We present a unique approach to ground truth representation for capturing more informative features. Finally, we incorporate a two-stage learning component into our framework to optimize estimation accuracy. Through evaluations on benchmark datasets, we demonstrate the effectiveness of our contributions and their practical relevance in healthcare applications. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/kateri25a.html https://proceedings.mlr.press/v259/kateri25a.html Timely detection and treatment are essential for maintaining eye health. Visual acuity (VA), which measures the clarity of vision at a distance, is a crucial metric for managing eye health. Machine learning (ML) techniques have been introduced to assist in VA measurement, potentially alleviating clinicians’ workloads. However, the inherent uncertainties in ML models make relying solely on them for VA prediction less than ideal. The VA prediction task involves multiple sources of uncertainty, requiring more robust approaches. A promising method is to build prediction sets or intervals rather than point estimates, offering coverage guarantees through techniques like conformal prediction and Probably Approximately Correct (PAC) prediction sets. Despite the potential, to date, these approaches have not been applied to the VA prediction task.To address this, we propose a method for deriving prediction intervals for estimating visual acuity from fundus images with a PAC guarantee. Our experimental results demonstrate that the PAC guarantees are upheld, with performance comparable to or better than that of two prior works that do not provide such guarantees. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/jang25a.html https://proceedings.mlr.press/v259/jang25a.html Surgical scene understanding is a key technical component for enabling intelligent and context aware systems that can transform various aspects of surgical interventions. In this work, we focus on the semantic segmentation task, propose a simple yet effective multi-modal (RGB and depth) training framework called SurgDepth, and show state-of-the-art (SOTA) results on all publicly available datasets applicable for this task. Unlike previous approaches, which either fine-tune SOTA segmentation models trained on natural images, or encode RGB or RGB-D information using RGB only pre-trained backbones, SurgDepth, which is built on top of Vision Transformers (ViTs), is designed to encode both RGB and depth information through a simple fusion mechanism. We conduct extensive experiments on benchmark datasets including EndoVis2022, AutoLapro, LapI2I and EndoVis2017 to verify the efficacy of SurgDepth. Specifically, SurgDepth achieves a new SOTA IoU of 0.86 on EndoVis 2022 SAR-RARP50 challenge and outperforms the current best method by at least 4%, using a shallow and compute efficient decoder consisting of ConvNeXt blocks. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/jamal25a.html https://proceedings.mlr.press/v259/jamal25a.html Chest X-ray imaging is a widely accessible and non-invasive diagnostic tool for detecting thoracic abnormalities. While numerous AI models assist radiologists in interpreting these images, most overlook patients’ historical data. To bridge this gap, we introduce Temporal MIMIC dataset, which integrates five years of patient history, including radiographic scans and reports from MIMIC-CXR and MIMIC-IV, encompassing 12,221 patients and thirteen pathologies. Building on this, we present HIST-AID, a framework that enhances automatic diagnostic accuracy using historical reports. HIST-AID emulates the radiologist’s comprehensive approach, leveraging historical data to improve diagnostic accuracy. Our experiments demonstrate significant improvements, with AUROC increasing by 6.56% and AUPRC by 9.51% compared to models that rely solely on radiographic scans. These gains were consistently observed across diverse demo- graphic groups, including variations in gender, age, and racial categories. We show that while recent data boost performance, older data may reduce accuracy due to changes in patient conditions. Our work paves the potential of incor- porating historical data for more reliable automatic diagnosis, providing critical support for clinical decision-making. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/huang25a.html https://proceedings.mlr.press/v259/huang25a.html CT cardiac imaging remains one of the most challenging visualization techniques among numerous CT organ imaging procedures. This is because of the dynamic nature of human hearts, constantly moving and pumping blood. Due to cardiac motions, CT scanners need to be capable of taking fast scans to capture a “snapshot” of a human heart. Other cardiac imaging challenges include contrast timing variations, radiation dose to patient bodies, limited temporal resolution, contrast agent allergies, and more. In this paper, we present a new latent diffusion model for 3D CT cardiac imaging where the model produces both image volumes and segmentation labels. The latent diffusion model is trained with distinct data augmentation techniques to enhance the variety of the generative data. This helps capture the dynamic nature of the cardiac images. The generative data are used in our Self-Supervised Learning (SSL) to pre-train our Deep Learning (DL) model. Furthermore, because our latent diffusion model produces both images and segmentation labels, our fine-tuning process takes advantage of the diffusion-generated images and labels in addition to the GT data. We run extensive experiments to show that the latent diffusion model and the SSL do help improve 3D CT cardiac image segmentation performance. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/hu25a.html https://proceedings.mlr.press/v259/hu25a.html Biomedical images are often high-resolution and multi-dimensional, presenting computational challenges for deep neural networks. These computational challenges are compounded when training transformers due to the self-attention operator, which scales quadratically with context length. Recent works have proposed alternatives to self-attention that scale more favorably with context length, alleviating these computational difficulties and potentially enabling more efficient application of transformers to large biomedical images. However, a systematic evaluation on this topic is lacking. In this study, we investigate the impact of context length on biomedical image analysis and we evaluate the performance of recently proposed substitutes for self-attention. We first curate a suite of biomedical imaging datasets, including 2D and 3D data for segmentation, denoising, and classification tasks. We then analyze the impact of context length on network performance using the Vision Transformer and Swin Transformer. Our findings reveal a strong relationship between context length and performance, particularly for pixel-level prediction tasks. Finally, we show that recent attention-free models demonstrate significant improvements in efficiency while maintaining comparable performance to self-attention-based models. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/hooper25a.html https://proceedings.mlr.press/v259/hooper25a.html Measurement quality assurance (QA) prac- tices play a key role in the safe use of Inten- sity Modulated Radiation Therapies (IMRT) for cancer treatment. These practices have re- duced measurement-based IMRT QA failure be- low 1%. However, these practices are time and labor intensive which can lead to delays in pa- tient care. In this study, we examine how con- formal prediction methodologies can be used to robustly triage plans. We propose a new training-aware conformal risk control method by combining the benefit of conformal risk con- trol and conformal training. We incorporate the decision-making thresholds based on the GPR, along with the risk functions used in clinical evaluation, into the design of the risk control framework. Our method achieves high sensitiv- ity and specificity and significantly reduces the number of plans needing measurement without generating a huge confidence interval. Our re- sults demonstrate the validity and applicabil- ity of conformal prediction methods for improv- ing efficiency and reducing the workload of the IMRT QA process. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/he25a.html https://proceedings.mlr.press/v259/he25a.html During surgical training, real-time feedback from trainers to trainees is important for preventing errors and enhancing long-term skill acquisition. Accurately predicting the effectiveness of this feedback, specifically whether it leads to a change in trainee behavior, is crucial for developing methods for improving surgical training and education. However, relying on human annotations to assess feedback effectiveness is laborious and prone to biases, underscoring the need for an automated, scalable, and objective method. Creating such an automated system poses challenges, as it requires an understanding of both the verbal feedback delivered by the trainer and the visual context of the real-time surgical scene. To address this, we propose a method that integrates information from transcribed verbal feedback and corresponding surgical video to predict feedback effectiveness. Our findings show that both transcribed feedback and surgical video are individually predictive of trainee behavior changes, and their combination achieves an AUROC of 0.70 ± 0.02, improving prediction accuracy by up to 6.6%. Additionally, we introduce self-supervised fine-tuning as a strategy for enhancing surgical video representation learning, which is scalable and further enhances prediction performance. Our results demonstrate the potential of multi-modal learning to advance the automated assessment of surgical feedback. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/gupta25a.html https://proceedings.mlr.press/v259/gupta25a.html Judgment biases in emergency triage can adversely affect patient outcomes. This study examines sex/gender biases using four advanced language models fine-tuned on real-world emergency department data. We introduce a novel approach based on the testing method, commonly used in hiring bias detection, by automatically altering triage notes to change patient sex references. Results indicate a significant bias: female patients are assigned lower severity ratings than male patients with identical clinical conditions. This bias is more pronounced with female nurses or when patients report higher pain levels but diminishes with increased nurse experience. Identifying these biases can inform interventions such as enhanced training, protocol updates, and machine learning tools to support clinical decision-making. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/guerra-adames25a.html https://proceedings.mlr.press/v259/guerra-adames25a.html The rise of foundation models, particularly large language models like ChatGPT, has revolutionized natural language processing and demonstrated remarkable generalization across numerous healthcare applications. Building on this success, foundation models for time series forecasting have emerged, offering new opportunities by leveraging pretraining on large-scale datasets. However, existing time series foundation models are pretrained with minimal clinical data, and their potentials for continuously recorded clinical time series, such as vital signs, remain largely under-explored. This motivates our endeavor to integrate time series foundation models with vital sign data to address critical clinical challenges, particularly in predicting patient deterioration. Through an extensive evaluation of various settings and configurations of these models, alongside comparisons with conventional forecasting models, we highlight the significant opportunities for improvement in developing clinically useful time series forecasting models. In a word, the “ChatGPT” moment for time series foundation models, in the typical clinical domain, is yet to come. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/gu25a.html https://proceedings.mlr.press/v259/gu25a.html RNA sequencing (RNA-seq) has become a key technology in precision medicine, especially for cancer prognosis. However, the high dimensionality of such data may restrict classic statistical methods, thus raising the need to learn dense representations from them. Transformers models have exhibited capacities in providing representations for long sequences and thus are well suited for transcriptomics data. In this paper, we develop a pre-trained transformer-based language model through self-supervised learning using bulk RNA-seq from both non-cancer and cancer tissues, following BERT’s masking method. By probing learned embeddings from the model or using parameter-efficient fine-tuning, we then build downstream models for cancer-type classification and survival-time prediction. Leveraging the TCGA dataset, we demonstrate the performance of our method, BulkRNABert, on both tasks, with significant improvement compared to state-of-the-art methods in the pan-cancer setting for classification and survival analysis. We also show the transfer-learning capabilities of the model in the survival analysis setting on unseen cohorts. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/gelard25a.html https://proceedings.mlr.press/v259/gelard25a.html Nursing notes, an important part of Electronic Health Records (EHRs), track a patient’s health during a care episode. Summarizing key information in nursing notes can help clinicians quickly understand patients’ conditions. However, existing summarization methods in the clinical setting, especially abstractive methods, have overlooked nursing notes and require reference summaries for training. We introduce QGSumm, a novel query-guided self-supervised domain adaptation approach for abstractive nursing note summarization. The method uses patient-related clinical queries for guidance, and hence does not need reference summaries for training. Through automatic experiments and manual evaluation by an expert clinician, we study our approach and other state-of-the-art Large Language Models (LLMs) for nursing note summarization. Our experiments show: 1) GPT-4 is competitive in maintaining information in the original nursing notes, 2) QGSumm can generate high-quality summaries with a good balance between recall of the original content and hallucination rate lower than other top methods. Ultimately, our work offers a new perspective on conditional text summarization, tailored to clinical applications. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/gao25a.html https://proceedings.mlr.press/v259/gao25a.html The task of predicting long-term patient outcomes using supervised machine learning is a challenging one, in part because of the high variance of each patient’s trajectory, which can result in the model over-fitting to the training data. Temporal difference (TD) learning, a common reinforcement learning technique, may reduce variance by generalising learning to the pattern of state transitions rather than terminal outcomes. However, in healthcare this method requires several strong assumptions about patient states, and there appears to be limited literature evaluating the performance of TD learning against traditional supervised learning methods for long-term health outcome prediction tasks. In this study, we define a framework for applying TD learning to real-time irregularly sampled time series data using a Semi-Markov Reward Process. We evaluate the model framework in predicting intensive care mortality and show that TD learning under this framework can result in improved model robustness compared to standard supervised learning methods, and that this robustness is maintained even when validated on external datasets. This approach may offer a more reliable method when learning to predict patient outcomes using high-variance irregular time series data. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/frost25a.html https://proceedings.mlr.press/v259/frost25a.html Digital health interventions (DHIs) and remote patient monitoring (RPM) have shown great potential in improving chronic disease management through personalized care. However, barriers like limited efficacy and workload concerns hinder adoption of existing DHIs, and limited sample sizes and lack of interpretability limit the effectiveness and adoption of purely black-box algorithmic DHIs. In this paper, we address these challenges by developing a pipeline for learning explainable treatment policies for RPM-enabled DHIs. We apply our approach in the real-world setting of RPM using a DHI to improve glycemic control of youth with type 1 diabetes. Our main contribution is to reveal the importance of \emph{clinical domain knowledge} in developing state and action representations for effective, efficient, and interpretable targeting policies. We observe that policies learned from clinician-informed representations are significantly more efficacious and efficient than policies learned from black-box representations. This work emphasizes the importance of collaboration between ML researchers and clinicians for developing effective DHIs in the real world. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/ferstad25a.html https://proceedings.mlr.press/v259/ferstad25a.html Reliable prediction of pediatric obesity can offer a valuable resource to providers, helping them engage in timely preventive interventions before the disease is established. Many efforts have been made to develop ML-based predictive models of obesity, and some studies have reported high predictive performances. However, no commonly used clinical decision support tool based on existing ML models currently exists. This study presents a novel end-to-end pipeline specifically designed for pediatric obesity prediction, which supports the entire process of data extraction, inference, and communication via an API or a user interface. While focusing only on routinely recorded data in pediatric electronic health records (EHRs), our pipeline uses a diverse expert-curated list of medical concepts to predict the 1-3 years risk of developing obesity. Furthermore, by using the Fast Healthcare Interoperability Resources (FHIR) standard in our design procedure, we specifically target facilitating low-effort integration of our pipeline with different EHR systems. In our experiments, we report the effectiveness of the predictive model as well as its alignment with the feedback from various stakeholders, including ML scientists, providers, health IT personnel, health administration representatives, and patient group representatives. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/fayyaz25a.html https://proceedings.mlr.press/v259/fayyaz25a.html Transformers have significantly advanced the modeling of Electronic Health Records (EHR), yet their deployment in real-world healthcare is limited by several key challenges. Firstly, the quadratic computational cost and insufficient context length of these models hinder hospitals’ ability in processing the extensive medical histories typical in EHR data. Additionally, existing models employ separate finetuning for each clinical task, complicating maintenance in healthcare environments. Moreover, these models focus exclusively on either clinical prediction or EHR forecasting, lacking proficiency in both tasks. To overcome these limitations, we introduce EhrMamba, a robust foundation model built on the Mamba architecture. EhrMamba can process sequences up to 300% times longer than previous models due to its linear computational cost. We also introduce a novel approach to Multitask Prompted Finetuning (MPF) for EHR data, which enables EhrMamba to simultaneously learn multiple clinical tasks in a single finetuning phase, significantly enhancing deployment and cross-task generalization. Furthermore, our model leverages the HL7 FHIR data standard to simplify integration into existing hospital systems. Alongside EhrMamba, we open-source Odyssey, a toolkit designed to support the development and deployment of EHR foundation models, with an emphasis on data standardization and interpretability. Our evaluations on the MIMIC-IV dataset demonstrate that EhrMamba advances state-of-the-art performance across 6 major clinical tasks and excels in EHR forecasting, marking a significant leap forward in the field. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/fallahpour25a.html https://proceedings.mlr.press/v259/fallahpour25a.html Large language models (LLMs) have revolutionized the question answering (QA) domain by achieving near-human performance across a broad range of tasks. Recent studies have suggested LLMs are capable of answering clinical questions and providing medical advice. Although LLMs’ answers must be reliable and safe, existing evaluations of medical QA systems often only focus on the accuracy of the content. However, a critical, underexplored aspect is whether variations in patient inquiries - rephrasing the same question - lead to inconsistent or unsafe LLM responses. We propose a new evaluation methodology leveraging synthetic question generation to rigorously assess the safety of LLMs in patient-facing medical QA. In benchmarking 8 LLMs, we observe a weak correlation between standard automated quality metrics and human evaluations, underscoring the need for enhanced sensitivity analysis in evaluating patient medical QA safety. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/diekmann25a.html https://proceedings.mlr.press/v259/diekmann25a.html In medical decision-making, clinicians must choose between different time-varying treatment strategies. Counterfactual prediction via g-computation enables comparison of alternative outcome distributions under such treatment strategies. While deep learning can better model high-dimensional data with complex temporal dependencies, incorporating model uncertainty into predicted conditional counterfactual distributions remains challenging. We propose a principled approach to model uncertainty in deep learning implementations of g-computations using approximate Bayesian posterior predictive distributions of counterfactual outcomes via variational dropout and deep ensembles. We evaluate these methods by comparing their counterfactual predictive calibration and performance in decision-making tasks, using two simulated datasets from mechanistic models and a real-world sepsis dataset. Our findings suggest that the proposed uncertainty quantification approach improves both calibration and decision-making performance, particularly in minimizing risks of worst-case adverse clinical outcomes under alternative dynamic treatment regimes. To our knowledge, this is the first work to propose and compare multiple uncertainty quantification methods in machine learning models of g-computation in estimating conditional treatment effects under dynamic treatment regimes. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/deng25a.html https://proceedings.mlr.press/v259/deng25a.html Medical machine learning algorithms are typically evaluated based on their object-level accuracy vs. that of skilled clinicians, a challenging bar since trained clinicians are usually better classifiers than ML models. However, this metric does not fully capture the realities and requirements of the actual clinical task: it neglects the fact that humans, even with perfect object-level accuracy, are subject to non-trivial error from the Poisson statistics of rare events, because clinical protocols often specify a remarkably small sample size due to the exigencies of clinical work. For example, to quantitate malaria on a thin blood film a clinician examines only 2000 red blood cells (0.0004 uL), which can yield large Poisson variation in the actual number of parasites present, so that a perfect human’s count can differ substantially from the true average load. In contrast, an ML system may be less accurate on an object detection level, but it may also have the option to examine much more blood (e.g. 0.1 uL, or 250x). Thus, while its parasite identification error is higher, the Poisson variability of its estimate is lower due to larger sample size. For both ML systems and humans, clinical performance depends on a combination of these two types of error. To qualify for clinical deployment, an ML system’s performance must match current standard of care, typically a demanding target. To achieve this, it may be possible to offset a system’s imperfect accuracy by increasing its sample size to reduce Poisson error, and thus attain the same net clinical performance as a perfectly accurate human limited by protocols with smaller sample size. In this paper, we analyse the mathematics of the relationship between Poisson error, classification error, and total error. This mathematical approach enables teams (software and hardware) optimizing ML systems to leverage a relative strength (larger sample sizes) to offset a relative weakness (classification accuracy). We illustrate the methods with two concrete examples: diagnosis and quantitation of malaria on blood films. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/delahunt25a.html https://proceedings.mlr.press/v259/delahunt25a.html In recent years, there has been growing interest in developing a non-invasive tool for detecting Alzheimer’s Disease (AD). Previous studies have shown that a single modality such as speech or eye-tracking (ET) data can be effective for classifying AD patients from healthy individuals. However, understanding the role of other modalities, and especially the integration of facial analysis with ET for enhancing dementia classification, remains under-explored. In this paper, we investigate whether we can leverage facial patterns in AD patients by building on EMOTION-FAN—a deep learning model initially developed for recognizing seven distinct human emotions, now fine-tuned for our facial analysis tasks. We also explore the efficacy of leveraging multimodal information by combining the results from the facial and ET data through a late fusion technique. Specifically, our approach uses a neural classifier to learn from raw ET data (VTNet) alongside the fine-tuned EMOTION-FAN model that learns from the facial data. Experimental results show that facial data gives superior results than ET data. Notably, we obtain higher scores when both modalities are combined, providing strong evidence that integrating multimodal data benefits performance on this task. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/chou25a.html https://proceedings.mlr.press/v259/chou25a.html Machine learning bias in mental health is becoming an increasingly pertinent challenge. Despite promising efforts indicating that multitask approaches often work better than unitask approaches, there is minimal work investigating the impact of multitask learning on performance and fairness in depression detection nor leveraged it to achieve fairer prediction outcomes. In this work, we undertake a systematic investigation of using a multitask approach to improve performance and fairness for depression detection. We propose a novel gender-based task-reweighting method using uncertainty grounded in how the PHQ-8 questionnaire is structured. Our results indicate that, although a multitask approach improves performance and fairness compared to a unitask approach, the results are not always consistent and we see evidence of negative transfer and a reduction in the Pareto frontier, which is concerning given the high-stake healthcare setting. Our proposed approach of gender-based reweighting with uncertainty improves performance and fairness and alleviates both challenges to a certain extent. Our findings on each PHQ-8 subitem task difficulty are also in agreement with the largest study conducted on the PHQ-8 subitem discrimination capacity, thus providing the very first tangible evidence linking ML findings with large-scale empirical population studies conducted on the PHQ-8. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/cheong25a.html https://proceedings.mlr.press/v259/cheong25a.html Accurate classification of sleep stages from less obtrusive sensor measurements such as the electrocardiogram (ECG) or photoplethysmogram (PPG) could enable important applications in sleep medicine. Existing approaches to this problem have typically used deep learning models designed and trained to operate on one or more specific input signals. However, the datasets used to develop these models often do not contain the same sets of input signals. Some signals, particularly PPG, are much less prevalent than others, and this has previously been addressed with techniques such as transfer learning. Additionally, only training on one or more fixed modalities precludes cross-modal information transfer from other sources, which has proved valuable in other problem domains. To address this, we introduce wav2sleep, a unified model designed to operate on variable sets of input signals during training and inference. After jointly training on over 10,000 overnight recordings from six publicly available polysomnography datasets, including SHHS and MESA, wav2sleep outperforms existing sleep stage classification models across test-time input combinations including ECG, PPG, and respiratory signals. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/carter25a.html https://proceedings.mlr.press/v259/carter25a.html In the aftermath of the COVID-19 pandemic and amid accelerating climate change, emerging infectious diseases, particularly those arising from zoonotic spillover, remain a global threat. Mpox (caused by the monkeypox virus) is a notable example of a zoonotic infection that often goes undiagnosed, especially as its rash progresses through stages, complicating detection across diverse populations with different presentations. In August 2024, the WHO Director-General declared the mpox outbreak a public health emergency of international concern for a second time. Despite the deployment of deep learning techniques for detecting diseases from skin lesion images, a robust and publicly accessible foundation model for mpox diagnosis is still lacking due to the unavailability of open-source mpox skin lesion images, multimodal clinical data, and specialized training pipelines. To address this gap, we propose MpoxVLM, a vision-language model (VLM) designed to detect mpox by analyzing both skin lesion images and patient clinical information. MpoxVLM integrates the CLIP visual encoder, an enhanced Vision Transformer (ViT) classifier for skin lesions, and LLaMA-2-7B models, pre-trained and fine-tuned on visual instruction-following question-answer pairs from our newly released mpox skin lesion dataset. Our work achieves 90.38% accuracy for mpox detection, offering a promising pathway to improve early diagnostic accuracy in combating mpox. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/cao25a.html https://proceedings.mlr.press/v259/cao25a.html We applied machine learning to the unmet medical need of rapid and accurate diagnosis and prognosis of acute infections and sepsis in emergency departments. Our solution consists of a Myrna Instrument and embedded TriVerity classifiers. The instrument measures abundances of 29 messenger RNAs in patient’s blood, subsequently used as features for machine learning. The classifiers convert the input features to an intuitive test report comprising the separate likelihoods of (1) a bacterial infection (2) a viral infection, and (3) severity (need for Intensive Care Unit-level care). In internal validation, the system achieved AUROC = 0.83 on the three-class disease diagnosis (bacterial, viral, or non-infected) and AUROC = 0.77 on binary prognosis of disease severity. The Myrna, TriVerity system was granted breakthrough device designation by the United States Food and Drug Administration (FDA). This engineering manuscript teaches the standard and novel machine learning methods used to translate an academic research concept to a clinical product aimed at improving patient care, and discusses lessons learned. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/buturovic25a.html https://proceedings.mlr.press/v259/buturovic25a.html Meningeal lymphatic vessels (MLVs) are responsible for the drainage of waste products from the human brain. An impairment in their functionality has been associated with aging as well as brain disorders like multiple sclerosis and Alzheimer’s disease. However, MLVs have only recently been described for the first time in magnetic resonance imaging (MRI), and their ramified structure renders manual segmentation particularly difficult. Further, as there is no consistent notion of their appearance, human-annotated MLV structures contain a high inter-rater variability that most automatic segmentation methods cannot take into account. In this work, we propose a new rater-aware training scheme for the popular nnUNet model, and we explore rater-based ensembling strategies for accurate and consistent segmentation of MLVs. This enables us to boost nnU-Net’s performance while obtaining explicit predictions in different annotation styles and a rater-based uncertainty estimation. Our final model, MLV2-Net, achieves a Dice similarity coefficient of 0.806 with respect to the human reference standard. The model further matches the human inter-rater reliability and replicates age-related associations with MLV volume. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/bongratz25a.html https://proceedings.mlr.press/v259/bongratz25a.html Early detection of heart block can prevent life-threatening outcomes in patients with cardiac conduction disorders. While 12-lead ECG interpretation is the clinical standard apparatus, this work investigates detecting heart block from the lead-I ECG signals, the lead available on commercial smartwatches. We evaluate two state-of-the-art architectures: residual neural network and transformer encoder, both trained in a self-supervised contrastive learning manner with a novel signal-continuity-based ECG view definition on a dataset of 3.6 million ECGs from Massachusetts General Hospital. These models learn efficient ECG representations, which are used for heart block detection via linear probing on the PTB-XL dataset, a public ECG resource. To provide performance benchmarks, we compare our self-supervised models to supervised adaptations of both models trained on 10.6 thousand single-lead PTB-XL ECGs. Our analysis evaluates the performance of each model using the area under the receiver-operating curve (AUC), sensitivity, and specificity. We observe improved performance from the self-supervised pretraining. Additionally, we demonstrate the robust generalizability of these models in scarce-data scenarios, maintaining consistent performance with a reduced number of labeled training examples. This study highlights the potential of self-supervised learning in lead-I ECG diagnostics, offering promising implications for clinical applications where labeled data is scarce. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/bergamaschi25a.html https://proceedings.mlr.press/v259/bergamaschi25a.html In this work we introduce Labrador, a pre-trained Transformer model for laboratory data. Labrador and BERT were pre-trained on a corpus of 100 million lab test results from electronic health records (EHRs) and evaluated on various downstream outcome prediction tasks. Both models demonstrate mastery of the pre-training task but neither consistently outperform XGBoost on downstream supervised tasks. Our ablation studies reveal that transfer learning shows limited effectiveness for BERT and achieves marginal success with Labrador. We explore the reasons for the failure of transfer learning and suggest that the data generating process underlying each patient cannot be characterized sufficiently using labs alone, among other factors. We encourage future work to focus on joint modeling of multiple EHR data categories and to include tree-based baselines in their evaluations. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/bellamy25a.html https://proceedings.mlr.press/v259/bellamy25a.html Electrical impedance tomography (EIT) aims to reconstruct the body’s internal electrical conductivity distribution from surface voltage measurements. This non-invasive, non-ionizing, and cost-effective technique is valuable for medical applications, offering potential for long- term monitoring of lung functionality and condition. However, existing methods of absolute EIT provide blurred tomograms that are difficult to interpret, do not resemble the human topography and are therefore of limited use for clinical applications. We propose RESIST, a new data-driven approach that integrates prior geometry of human bodies and conductivity information by combining an implicit neural network with a transformer model in an encoder-decoder framework. RESIST maps simultaneous EIT measurements from multiple body levels to conductivity values at any coordinate in a 3D body volume. We train RESIST on simulated EIT measurements based on 3D human body CT segmentations. We find that it is robust against distortions in the signal and exact placement of electrodes, correctly infers interpolation laws, and generalizes to real EIT measurements in humans. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/becker25a.html https://proceedings.mlr.press/v259/becker25a.html Spatial transcriptomics (ST) technologies enable the mapping of gene and protein abundance within specific tissue architectures, representing a significant advancement over conventional bulk analyses that can obscure critical prognostic markers tied to spatial contexts. Expanding these analyses to three dimensions (3D) can further uncover intricate biomolecular phenomena that may be truncated or missed in two-dimensional (2D) studies. However, the widespread application of 3D ST profiling is limited by high costs and logistical challenges. Deep learning-based inference of ST data from routine histopathological staining offers a cost-effective alternative, allowing for the exploration of histologically associated biological pathways in 3D and enhancing our ability to detect structures linked to tumor progression. In this proof-of-concept study, we employed deep learning models to infer ST data from routine histopathology for 10 colorectal cancer patients, with 10 serial sections analyzed per patient. Our downstream analyses revealed several key instances where 3D approaches provided enhanced insights into cellular phenomena compared to traditional 2D methods. These findings lay the groundwork for future research aimed at leveraging these methods to investigate subtle 3D biomarkers associated with tumor metastasis and recurrence. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/azher25a.html https://proceedings.mlr.press/v259/azher25a.html Electroencephalography (EEG) plays a critical role in the monitoring and diagnosis of neurological disorders, particularly in detecting seizures and other harmful brain activities. However, interpreting EEG signals is a complex task that often suffers from high variability and subjectivity among clinical experts. This study introduces the BiG-WaR architecture, a comprehensive multimodal framework designed to classify harmful brain activity using EEG signals. BiG-WaR combines several neural network models, including BiLSTM, GNN, WaveNet, and ResNet, to effectively leverage spatial and temporal dynamics inherent in EEG data. Our approach integrates curriculum learning and appropriate data preprocessing to address the challenges of EEG analysis, such as high variability and the need for robust feature extraction. Initial results demonstrate that BiG-WaR framework is a robust benchmark, enhancing reliability and interpretability—critical factors for clinical adoption—by integrating attention mechanisms and gradient-weighted class activation mappings to provide insights into model decisions. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/azhar25b.html https://proceedings.mlr.press/v259/azhar25b.html This study systematically compares vision transformers (ViTs) and residual neural networks (ResNets) for slide-level histopathology analysis using self-supervised learning (SSL) strategies SimCLR and DINO, alongside ImageNet-initialized counterparts. Binary and multiclass classification were evaluated on the TCGA-CRCk and PANDA datasets using the multiple instance learning paradigm. GradCAM and attention rollout were applied for visual explainability for ResNets and ViTs, respectively. The results showed that SimCLR excelled with smaller architectures, while DINO performed better with larger ones. ResNets underperformed with DINO regardless of architecture size, whereas ViTs excelled on the PANDA dataset but required scaling to perform well on TCGA-CRCk, a more complex dataset. An emerging self-supervision regime and encoder fit suggests that DINO-ViTs and SimCLR-ResNets are well-suited pairs, though the effects of SSL methods on encoders remain unclear. ViTs demonstrated superior scalability and interpretability compared to ResNets, but further research is needed to better understand the interplay between SSL methods and encoders. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/azhar25a.html https://proceedings.mlr.press/v259/azhar25a.html The t-Distributed Stochastic Neighbor Embedding (t-SNE) has emerged as a popular dimensionality reduction technique for visualizing high-dimensional data. It computes pairwise similarities between data points by default using an RBF kernel and random initialization (in low-dimensional space), which successfully captures the overall structure but may struggle to preserve the local structure efficiently. This research proposes a novel approach called the Modified Isolation Kernel (MIK) as an alternative to the Gaussian kernel, which is built upon the concept of the Isolation Kernel. MIK uses adaptive density estimation to capture local structures more accurately and integrates robustness measures. It also assigns higher similarity values to nearby points and lower values to distant points. Comparative research using the normal Gaussian kernel, the isolation kernel, and several initialization techniques, including random, PCA, and random walk initializations, are used to assess the proposed approach (MIK). Additionally, we compare the computational efficiency of all $3$ kernels with $3$ different initialization methods. Our experimental results demonstrate several advantages of the proposed kernel (MIK) and initialization method selection. It exhibits improved preservation of the local and global structure and enables better visualization of clusters and subclusters in the embedded space. These findings contribute to advancing dimensionality reduction techniques and provide researchers and practitioners with an effective tool for data exploration, visualization, and analysis in various domains. Mon, 17 Feb 2025 00:00:00 +0000 https://proceedings.mlr.press/v259/ali25a.html https://proceedings.mlr.press/v259/ali25a.html