SleepLM: Natural-Language Intelligence for Human Sleep

Zongzhe Xu, Zitao Shuai, Eideen Mozaffari, Ravi Shankar Aysola, Rajesh Kumar, Yuzhe Yang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:141707-141738, 2026.

Abstract

We present SleepLM, a family of sleep-language foundation models that enable human sleep alignment, interpretation, and interaction with natural language. Despite the critical role of sleep, learning-based sleep analysis systems operate in closed label spaces (e.g., predefined stages or events) and fail to describe, query, or generalize to novel sleep phenomena. SleepLM bridges natural language and multimodal polysomnography, enabling language-grounded representations of sleep physiology. To support this alignment, we introduce a multilevel sleep caption generation pipeline that enables the curation of the first large-scale sleep-text dataset, comprising over 100K hours of data from more than 10,000 individuals. Furthermore, we present a unified pretraining objective that combines contrastive alignment, caption generation, and signal reconstruction to better capture physiological fidelity and cross-modal interactions. Extensive experiments on real-world sleep understanding tasks verify that SleepLM outperforms state-of-the-art in zero-shot and few-shot learning, cross-modal retrieval, and sleep captioning. Importantly, SleepLM also exhibits intriguing capabilities including language-guided event localization, targeted insight generation, and zero-shot generalization to unseen tasks. To support reproducibility and future work, we open-source the captioning pipeline, pretrained checkpoints, and the model architectures at https://github.com/yang-ai-lab/SleepLM.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-xu26at, title = {{S}leep{LM}: Natural-Language Intelligence for Human Sleep}, author = {Xu, Zongzhe and Shuai, Zitao and Mozaffari, Eideen and Aysola, Ravi Shankar and Kumar, Rajesh and Yang, Yuzhe}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {141707--141738}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/xu26at/xu26at.pdf}, url = {https://proceedings.mlr.press/v306/xu26at.html}, abstract = {We present SleepLM, a family of sleep-language foundation models that enable human sleep alignment, interpretation, and interaction with natural language. Despite the critical role of sleep, learning-based sleep analysis systems operate in closed label spaces (e.g., predefined stages or events) and fail to describe, query, or generalize to novel sleep phenomena. SleepLM bridges natural language and multimodal polysomnography, enabling language-grounded representations of sleep physiology. To support this alignment, we introduce a multilevel sleep caption generation pipeline that enables the curation of the first large-scale sleep-text dataset, comprising over 100K hours of data from more than 10,000 individuals. Furthermore, we present a unified pretraining objective that combines contrastive alignment, caption generation, and signal reconstruction to better capture physiological fidelity and cross-modal interactions. Extensive experiments on real-world sleep understanding tasks verify that SleepLM outperforms state-of-the-art in zero-shot and few-shot learning, cross-modal retrieval, and sleep captioning. Importantly, SleepLM also exhibits intriguing capabilities including language-guided event localization, targeted insight generation, and zero-shot generalization to unseen tasks. To support reproducibility and future work, we open-source the captioning pipeline, pretrained checkpoints, and the model architectures at https://github.com/yang-ai-lab/SleepLM.} }
Endnote
%0 Conference Paper %T SleepLM: Natural-Language Intelligence for Human Sleep %A Zongzhe Xu %A Zitao Shuai %A Eideen Mozaffari %A Ravi Shankar Aysola %A Rajesh Kumar %A Yuzhe Yang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-xu26at %I PMLR %P 141707--141738 %U https://proceedings.mlr.press/v306/xu26at.html %V 306 %X We present SleepLM, a family of sleep-language foundation models that enable human sleep alignment, interpretation, and interaction with natural language. Despite the critical role of sleep, learning-based sleep analysis systems operate in closed label spaces (e.g., predefined stages or events) and fail to describe, query, or generalize to novel sleep phenomena. SleepLM bridges natural language and multimodal polysomnography, enabling language-grounded representations of sleep physiology. To support this alignment, we introduce a multilevel sleep caption generation pipeline that enables the curation of the first large-scale sleep-text dataset, comprising over 100K hours of data from more than 10,000 individuals. Furthermore, we present a unified pretraining objective that combines contrastive alignment, caption generation, and signal reconstruction to better capture physiological fidelity and cross-modal interactions. Extensive experiments on real-world sleep understanding tasks verify that SleepLM outperforms state-of-the-art in zero-shot and few-shot learning, cross-modal retrieval, and sleep captioning. Importantly, SleepLM also exhibits intriguing capabilities including language-guided event localization, targeted insight generation, and zero-shot generalization to unseen tasks. To support reproducibility and future work, we open-source the captioning pipeline, pretrained checkpoints, and the model architectures at https://github.com/yang-ai-lab/SleepLM.
APA
Xu, Z., Shuai, Z., Mozaffari, E., Aysola, R.S., Kumar, R. & Yang, Y.. (2026). SleepLM: Natural-Language Intelligence for Human Sleep. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:141707-141738 Available from https://proceedings.mlr.press/v306/xu26at.html.

Related Material