Olivia: Harmonizing Time Series Foundation Models with Power Spectral Density

Jingru Fei, Kun Yi, Alex Xing Wang, Qingsong Wen, Xiangxiang Zhu, Wei Fan
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:29833-29861, 2026.

Abstract

Time series foundation models rely on large-scale pretraining over diverse datasets across domains, yet their heterogeneity in temporal patterns could hinder the effectiveness of training and learning transferable time series representations. Inspired a fundamental concept, normalized power spectral density (PSD) in signal processing, we assume harmonizing datasets via PSDs in the spectral domain could reduce mismatches and enhance pretraining. We then go beyond the direct intractable minimization optimization and innovatively reformulate it as a principled harmonization approach. Specifically, we propose Harmonizer, a module that reshapes spectral structures and implicitly harmonizing PSDs across datasets, which theoretically corresponds to a shared reparameterization of second-order temporal correlations. Our theoretical analysis further reveals token interactions with Harmonizer can be efficiently mediated by a compact set of resonators, motivating a HarmonicAttention design that performs self-attention in a low-dimensional interaction space. Then, we propose Olivia, a novel time series foundation model built upon these harmonization mechanisms. Extensive experiments on several large-scale benchmarks (TSLib, GIFT-Eval, and GluonTS), demonstrate Olivia consistently achieves state-of-the-art performance under zero-shot, few-shot, and full-shot forecasting scenarios. Our code is at https://github.com/TSTS13/Olivia.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-fei26b, title = {Olivia: Harmonizing Time Series Foundation Models with Power Spectral Density}, author = {Fei, Jingru and Yi, Kun and Wang, Alex Xing and Wen, Qingsong and Zhu, Xiangxiang and Fan, Wei}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {29833--29861}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/fei26b/fei26b.pdf}, url = {https://proceedings.mlr.press/v306/fei26b.html}, abstract = {Time series foundation models rely on large-scale pretraining over diverse datasets across domains, yet their heterogeneity in temporal patterns could hinder the effectiveness of training and learning transferable time series representations. Inspired a fundamental concept, normalized power spectral density (PSD) in signal processing, we assume harmonizing datasets via PSDs in the spectral domain could reduce mismatches and enhance pretraining. We then go beyond the direct intractable minimization optimization and innovatively reformulate it as a principled harmonization approach. Specifically, we propose Harmonizer, a module that reshapes spectral structures and implicitly harmonizing PSDs across datasets, which theoretically corresponds to a shared reparameterization of second-order temporal correlations. Our theoretical analysis further reveals token interactions with Harmonizer can be efficiently mediated by a compact set of resonators, motivating a HarmonicAttention design that performs self-attention in a low-dimensional interaction space. Then, we propose Olivia, a novel time series foundation model built upon these harmonization mechanisms. Extensive experiments on several large-scale benchmarks (TSLib, GIFT-Eval, and GluonTS), demonstrate Olivia consistently achieves state-of-the-art performance under zero-shot, few-shot, and full-shot forecasting scenarios. Our code is at https://github.com/TSTS13/Olivia.} }
Endnote
%0 Conference Paper %T Olivia: Harmonizing Time Series Foundation Models with Power Spectral Density %A Jingru Fei %A Kun Yi %A Alex Xing Wang %A Qingsong Wen %A Xiangxiang Zhu %A Wei Fan %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-fei26b %I PMLR %P 29833--29861 %U https://proceedings.mlr.press/v306/fei26b.html %V 306 %X Time series foundation models rely on large-scale pretraining over diverse datasets across domains, yet their heterogeneity in temporal patterns could hinder the effectiveness of training and learning transferable time series representations. Inspired a fundamental concept, normalized power spectral density (PSD) in signal processing, we assume harmonizing datasets via PSDs in the spectral domain could reduce mismatches and enhance pretraining. We then go beyond the direct intractable minimization optimization and innovatively reformulate it as a principled harmonization approach. Specifically, we propose Harmonizer, a module that reshapes spectral structures and implicitly harmonizing PSDs across datasets, which theoretically corresponds to a shared reparameterization of second-order temporal correlations. Our theoretical analysis further reveals token interactions with Harmonizer can be efficiently mediated by a compact set of resonators, motivating a HarmonicAttention design that performs self-attention in a low-dimensional interaction space. Then, we propose Olivia, a novel time series foundation model built upon these harmonization mechanisms. Extensive experiments on several large-scale benchmarks (TSLib, GIFT-Eval, and GluonTS), demonstrate Olivia consistently achieves state-of-the-art performance under zero-shot, few-shot, and full-shot forecasting scenarios. Our code is at https://github.com/TSTS13/Olivia.
APA
Fei, J., Yi, K., Wang, A.X., Wen, Q., Zhu, X. & Fan, W.. (2026). Olivia: Harmonizing Time Series Foundation Models with Power Spectral Density. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:29833-29861 Available from https://proceedings.mlr.press/v306/fei26b.html.

Related Material