Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecasting

Huu Hiep Nguyen, Minh Hoang Nguyen, Dung Nguyen, Hung Le
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:4213-4221, 2026.

Abstract

Multimodal time series forecasting is crucial in real-world applications, where decisions depend on both numerical data and contextual signals. The core challenge is to effectively combine temporal numerical patterns with the context embedded in other modalities, such as text. While most existing methods align textual features with time-series patterns one step at a time, they neglect the multiscale temporal influences of contextual information such as time-series cycles and dynamic shifts. This mismatch between local alignment and global textual context can be addressed by spectral decomposition, which separates time series into frequency components capturing both short-term changes and long-term trends. In this paper, we propose SpecTF, a simple yet effective framework that integrates the effect of textual data on time series in the frequency domain. Our method extracts textual embeddings, projects them into the frequency domain, and fuses them with the time series’ spectral components using a lightweight cross-attention mechanism. This adaptively reweights frequency bands based on textual relevance before mapping the results back to the temporal domain for predictions. Experimental results demonstrate that SpecTF significantly outperforms state-of-the-art models across diverse multi-modal time series datasets while utilizing considerably fewer parameters. Code is available at \url{https://github.com/hiepnh137/SpecTF}.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-nguyen26d, title = { Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecasting }, author = {Nguyen, Huu Hiep and Nguyen, Minh Hoang and Nguyen, Dung and Le, Hung}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {4213--4221}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/nguyen26d/nguyen26d.pdf}, url = {https://proceedings.mlr.press/v300/nguyen26d.html}, abstract = { Multimodal time series forecasting is crucial in real-world applications, where decisions depend on both numerical data and contextual signals. The core challenge is to effectively combine temporal numerical patterns with the context embedded in other modalities, such as text. While most existing methods align textual features with time-series patterns one step at a time, they neglect the multiscale temporal influences of contextual information such as time-series cycles and dynamic shifts. This mismatch between local alignment and global textual context can be addressed by spectral decomposition, which separates time series into frequency components capturing both short-term changes and long-term trends. In this paper, we propose SpecTF, a simple yet effective framework that integrates the effect of textual data on time series in the frequency domain. Our method extracts textual embeddings, projects them into the frequency domain, and fuses them with the time series’ spectral components using a lightweight cross-attention mechanism. This adaptively reweights frequency bands based on textual relevance before mapping the results back to the temporal domain for predictions. Experimental results demonstrate that SpecTF significantly outperforms state-of-the-art models across diverse multi-modal time series datasets while utilizing considerably fewer parameters. Code is available at \url{https://github.com/hiepnh137/SpecTF}. } }
Endnote
%0 Conference Paper %T Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecasting %A Huu Hiep Nguyen %A Minh Hoang Nguyen %A Dung Nguyen %A Hung Le %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-nguyen26d %I PMLR %P 4213--4221 %U https://proceedings.mlr.press/v300/nguyen26d.html %V 300 %X Multimodal time series forecasting is crucial in real-world applications, where decisions depend on both numerical data and contextual signals. The core challenge is to effectively combine temporal numerical patterns with the context embedded in other modalities, such as text. While most existing methods align textual features with time-series patterns one step at a time, they neglect the multiscale temporal influences of contextual information such as time-series cycles and dynamic shifts. This mismatch between local alignment and global textual context can be addressed by spectral decomposition, which separates time series into frequency components capturing both short-term changes and long-term trends. In this paper, we propose SpecTF, a simple yet effective framework that integrates the effect of textual data on time series in the frequency domain. Our method extracts textual embeddings, projects them into the frequency domain, and fuses them with the time series’ spectral components using a lightweight cross-attention mechanism. This adaptively reweights frequency bands based on textual relevance before mapping the results back to the temporal domain for predictions. Experimental results demonstrate that SpecTF significantly outperforms state-of-the-art models across diverse multi-modal time series datasets while utilizing considerably fewer parameters. Code is available at \url{https://github.com/hiepnh137/SpecTF}.
APA
Nguyen, H.H., Nguyen, M.H., Nguyen, D. & Le, H.. (2026). Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecasting . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:4213-4221 Available from https://proceedings.mlr.press/v300/nguyen26d.html.

Related Material