Video-based Disease Progression Simulation

Modeling disease progression is crucial for improving the quality and efficacy of clinical diagnosis and prognosis, but it is often hindered by a lack of longitudinal medical image monitoring for individual patients. To address this challenge, we propose MedDream, the first video-based disease progression framework that enables controlled manipulation of disease-related image and video features, allowing precise, and personalized simulations of disease progression. Our approach begins by disease trajectory description recaptioning. Next, a controllable multi-round diffusion model simulates the disease progression state for each patient, creating realistic intermediate disease state sequences. Finally, a diffusion-based video transition generation model interpolates disease progression between these states. We validate our framework across three medical imaging domains: chest X-ray, fundus photography, and skin image. Our results demonstrate that MedDream significantly outperforms baseline models in generating coherent and clinically plausible disease trajectories. Two user studies by veteran physicians provide further validation into the clinical relevance of the generated sequences. MedDream has the potential to assist healthcare providers in modeling disease trajectories, interpolating missing medical image data, and enhancing medical education through realistic, dynamic visualizations of disease progression.