OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data

Patrick Langer, Thomas Kaar, Max Rosenblattl, Maxwell A Xu, Winnie Chow, Martin Maritsch, Robert Jakob, Ning Wang, Juncheng Liu, Aradhana Verma, Brian Han, Daniel Seung Kim, Henry Chubb, Scott R. Ceresnak, Aydin Zahedivash, Alexander T Sandhu, Fatima Rodriguez, Daniel Mcduff, Elgar Fleisch, Oliver Oppers Aalami, Filipe Barata, Paul Schmiedmayer
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:62642-62676, 2026.

Abstract

Large Language Models (LLMs) have shown strong capabilities in interpreting multimodal data but remain limited in handling time-series data. Addressing this gap could help translating longitudinal and wearable data into actionable insights and patient-facing digital health applications. We propose OpenTSLM, an open-source family of Time Series Language Models (TSLMs) integrating time-series as a native modality into pretrained LLMs, enabling natural-language prompting and reasoning over multiple time-series via either soft-prompting (OpenTSLM-SoftPrompt) or cross-attention (OpenTSLM-Flamingo). We introduce four datasets: M4-captioning, HAR-CoT (human activity recognition), Sleep-CoT (sleep staging), and ECG-QA-CoT (electrocardiogram question answering). Across tasks, OpenTSLM models outperform baselines like GPT-4o, reaching F1 scores of 69.88% in sleep staging and 67.64% in HAR; OpenTSLM-Flamingo also scales more efficiently in memory with longer or multiple time series. Expert evaluations with cardiologists show that OpenTSLM models exhibit strong reasoning capabilities on raw ECG data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-langer26a, title = {{O}pen{TSLM}: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data}, author = {Langer, Patrick and Kaar, Thomas and Rosenblattl, Max and Xu, Maxwell A and Chow, Winnie and Maritsch, Martin and Jakob, Robert and Wang, Ning and Liu, Juncheng and Verma, Aradhana and Han, Brian and Kim, Daniel Seung and Chubb, Henry and Ceresnak, Scott R. and Zahedivash, Aydin and Sandhu, Alexander T and Rodriguez, Fatima and Mcduff, Daniel and Fleisch, Elgar and Aalami, Oliver Oppers and Barata, Filipe and Schmiedmayer, Paul}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {62642--62676}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/langer26a/langer26a.pdf}, url = {https://proceedings.mlr.press/v306/langer26a.html}, abstract = {Large Language Models (LLMs) have shown strong capabilities in interpreting multimodal data but remain limited in handling time-series data. Addressing this gap could help translating longitudinal and wearable data into actionable insights and patient-facing digital health applications. We propose OpenTSLM, an open-source family of Time Series Language Models (TSLMs) integrating time-series as a native modality into pretrained LLMs, enabling natural-language prompting and reasoning over multiple time-series via either soft-prompting (OpenTSLM-SoftPrompt) or cross-attention (OpenTSLM-Flamingo). We introduce four datasets: M4-captioning, HAR-CoT (human activity recognition), Sleep-CoT (sleep staging), and ECG-QA-CoT (electrocardiogram question answering). Across tasks, OpenTSLM models outperform baselines like GPT-4o, reaching F1 scores of 69.88% in sleep staging and 67.64% in HAR; OpenTSLM-Flamingo also scales more efficiently in memory with longer or multiple time series. Expert evaluations with cardiologists show that OpenTSLM models exhibit strong reasoning capabilities on raw ECG data.} }
Endnote
%0 Conference Paper %T OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data %A Patrick Langer %A Thomas Kaar %A Max Rosenblattl %A Maxwell A Xu %A Winnie Chow %A Martin Maritsch %A Robert Jakob %A Ning Wang %A Juncheng Liu %A Aradhana Verma %A Brian Han %A Daniel Seung Kim %A Henry Chubb %A Scott R. Ceresnak %A Aydin Zahedivash %A Alexander T Sandhu %A Fatima Rodriguez %A Daniel Mcduff %A Elgar Fleisch %A Oliver Oppers Aalami %A Filipe Barata %A Paul Schmiedmayer %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-langer26a %I PMLR %P 62642--62676 %U https://proceedings.mlr.press/v306/langer26a.html %V 306 %X Large Language Models (LLMs) have shown strong capabilities in interpreting multimodal data but remain limited in handling time-series data. Addressing this gap could help translating longitudinal and wearable data into actionable insights and patient-facing digital health applications. We propose OpenTSLM, an open-source family of Time Series Language Models (TSLMs) integrating time-series as a native modality into pretrained LLMs, enabling natural-language prompting and reasoning over multiple time-series via either soft-prompting (OpenTSLM-SoftPrompt) or cross-attention (OpenTSLM-Flamingo). We introduce four datasets: M4-captioning, HAR-CoT (human activity recognition), Sleep-CoT (sleep staging), and ECG-QA-CoT (electrocardiogram question answering). Across tasks, OpenTSLM models outperform baselines like GPT-4o, reaching F1 scores of 69.88% in sleep staging and 67.64% in HAR; OpenTSLM-Flamingo also scales more efficiently in memory with longer or multiple time series. Expert evaluations with cardiologists show that OpenTSLM models exhibit strong reasoning capabilities on raw ECG data.
APA
Langer, P., Kaar, T., Rosenblattl, M., Xu, M.A., Chow, W., Maritsch, M., Jakob, R., Wang, N., Liu, J., Verma, A., Han, B., Kim, D.S., Chubb, H., Ceresnak, S.R., Zahedivash, A., Sandhu, A.T., Rodriguez, F., Mcduff, D., Fleisch, E., Aalami, O.O., Barata, F. & Schmiedmayer, P.. (2026). OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:62642-62676 Available from https://proceedings.mlr.press/v306/langer26a.html.

Related Material