Rethinking Multimodal Time-Series Forecasting Evaluation

Haoxin Liu, Yichen Zhou, Rajat Sen, B. Aditya Prakash, Abhimanyu Das
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:80183-80283, 2026.

Abstract

We introduce a new context-enriched, multimodal time series forecasting benchmark TimesX. TimesX contains a wide selection of high-quality real-world time series with diverse domains and textual contexts obtained from an automated data generation pipeline, which helps address three main issues of existing multimodal forecasting benchmarks: (1) poor generalization due to the small scale and synthetic nature of benchmark data, (2) very limited types of textual contexts in the benchmarks, and (3) an inability to mitigate data leakage in evaluation. We conduct a thorough empirical study of zero-shot multimodal forecasting approaches on TimesX. Our results suggest that many approaches that perform well on existing benchmarks may fail on TimesX. In contrast, simple ensemble methods that leverage rich textual context accompanying time-series can outperform strong baselines on the TimesX benchmark.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-liu26go, title = {Rethinking Multimodal Time-Series Forecasting Evaluation}, author = {Liu, Haoxin and Zhou, Yichen and Sen, Rajat and Prakash, B. Aditya and Das, Abhimanyu}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {80183--80283}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/liu26go/liu26go.pdf}, url = {https://proceedings.mlr.press/v306/liu26go.html}, abstract = {We introduce a new context-enriched, multimodal time series forecasting benchmark TimesX. TimesX contains a wide selection of high-quality real-world time series with diverse domains and textual contexts obtained from an automated data generation pipeline, which helps address three main issues of existing multimodal forecasting benchmarks: (1) poor generalization due to the small scale and synthetic nature of benchmark data, (2) very limited types of textual contexts in the benchmarks, and (3) an inability to mitigate data leakage in evaluation. We conduct a thorough empirical study of zero-shot multimodal forecasting approaches on TimesX. Our results suggest that many approaches that perform well on existing benchmarks may fail on TimesX. In contrast, simple ensemble methods that leverage rich textual context accompanying time-series can outperform strong baselines on the TimesX benchmark.} }
Endnote
%0 Conference Paper %T Rethinking Multimodal Time-Series Forecasting Evaluation %A Haoxin Liu %A Yichen Zhou %A Rajat Sen %A B. Aditya Prakash %A Abhimanyu Das %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-liu26go %I PMLR %P 80183--80283 %U https://proceedings.mlr.press/v306/liu26go.html %V 306 %X We introduce a new context-enriched, multimodal time series forecasting benchmark TimesX. TimesX contains a wide selection of high-quality real-world time series with diverse domains and textual contexts obtained from an automated data generation pipeline, which helps address three main issues of existing multimodal forecasting benchmarks: (1) poor generalization due to the small scale and synthetic nature of benchmark data, (2) very limited types of textual contexts in the benchmarks, and (3) an inability to mitigate data leakage in evaluation. We conduct a thorough empirical study of zero-shot multimodal forecasting approaches on TimesX. Our results suggest that many approaches that perform well on existing benchmarks may fail on TimesX. In contrast, simple ensemble methods that leverage rich textual context accompanying time-series can outperform strong baselines on the TimesX benchmark.
APA
Liu, H., Zhou, Y., Sen, R., Prakash, B.A. & Das, A.. (2026). Rethinking Multimodal Time-Series Forecasting Evaluation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:80183-80283 Available from https://proceedings.mlr.press/v306/liu26go.html.

Related Material