[edit]
ST2S-rPPG: A Spatiotemporal Two-Stage Learning Approach for Pulse Estimation Using Video
Proceedings of the 4th Machine Learning for Health Symposium, PMLR 259:550-562, 2025.
Abstract
Remote physiological monitoring presents an opportunity to enhance patient care, particularly in scenarios where traditional monitoring methods are impractical or unavailable. Heart rate, being a principal indicator of health, has been a focal point of video-based monitoring systems. Despite significant advancements in remote photoplethysmography technology, several challenges persist, including motion artifacts, data homogeneity and availability, which impact the accuracy and reliability of such solutions. In this work, we introduce a novel framework aimed at addressing these challenges, ST2S-rPPG. Our methodology involves a stabilization method to mitigate motion artifacts. We propose a spatiotemporal representation of video data, which captures predictive available information in the video and assists in transforming the input video. We present a unique approach to ground truth representation for capturing more informative features. Finally, we incorporate a two-stage learning component into our framework to optimize estimation accuracy. Through evaluations on benchmark datasets, we demonstrate the effectiveness of our contributions and their practical relevance in healthcare applications.