Bi-Level One-Shot Architecture Search for Probabilistic Time Series Forecasting

Jonas Seng, Fabian Kalter, Zhongjie Yu, Fabrizio Ventola, Kristian Kersting
Proceedings of the Third International Conference on Automated Machine Learning, PMLR 256:10/1-20, 2024.

Abstract

Time series forecasting is ubiquitous in many disciplines. A recent hybrid architecture named predictive Whittle networks (PWNs) tackles this task by employing two distinct modules, a tractable probabilistic model and a neural forecaster, with the former guiding the latter by providing likelihoods about predictions during training. Although PWNs achieve state-of-the-art accuracy, finding the optimal type of probabilistic model and neural forecaster (macro-architecture search) and the architecture of each module (micro-architecture search) of such hybrid models remains difficult and time-consuming. Current one-shot neural architecture search (NAS) methods approach this challenge by focusing on either the micro or the macro aspect, overlooking mutual impact, and could attain the overall optimization only sequentially. To overcome these limitations, we introduce a bi-level one-shot NAS method that optimizes such hybrid architectures simultaneously, leveraging the relationships between the micro and the macro architectural levels. We empirically demonstrate that the hybrid architectures found by our method outperform human-designed and overparameterized ones on various challenging datasets. Furthermore, we unveil insights about underlying connections between architectural choices and temporal features.

Cite this Paper

BibTeX
@InProceedings{pmlr-v256-seng24a, title = {Bi-Level One-Shot Architecture Search for Probabilistic Time Series Forecasting}, author = {Seng, Jonas and Kalter, Fabian and Yu, Zhongjie and Ventola, Fabrizio and Kersting, Kristian}, booktitle = {Proceedings of the Third International Conference on Automated Machine Learning}, pages = {10/1--20}, year = {2024}, editor = {Eggensperger, Katharina and Garnett, Roman and Vanschoren, Joaquin and Lindauer, Marius and Gardner, Jacob R.}, volume = {256}, series = {Proceedings of Machine Learning Research}, month = {09--12 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v256/main/assets/seng24a/seng24a.pdf}, url = {https://proceedings.mlr.press/v256/seng24a.html}, abstract = {Time series forecasting is ubiquitous in many disciplines. A recent hybrid architecture named predictive Whittle networks (PWNs) tackles this task by employing two distinct modules, a tractable probabilistic model and a neural forecaster, with the former guiding the latter by providing likelihoods about predictions during training. Although PWNs achieve state-of-the-art accuracy, finding the optimal type of probabilistic model and neural forecaster (macro-architecture search) and the architecture of each module (micro-architecture search) of such hybrid models remains difficult and time-consuming. Current one-shot neural architecture search (NAS) methods approach this challenge by focusing on either the micro or the macro aspect, overlooking mutual impact, and could attain the overall optimization only sequentially. To overcome these limitations, we introduce a bi-level one-shot NAS method that optimizes such hybrid architectures simultaneously, leveraging the relationships between the micro and the macro architectural levels. We empirically demonstrate that the hybrid architectures found by our method outperform human-designed and overparameterized ones on various challenging datasets. Furthermore, we unveil insights about underlying connections between architectural choices and temporal features.} }
Endnote
%0 Conference Paper %T Bi-Level One-Shot Architecture Search for Probabilistic Time Series Forecasting %A Jonas Seng %A Fabian Kalter %A Zhongjie Yu %A Fabrizio Ventola %A Kristian Kersting %B Proceedings of the Third International Conference on Automated Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Katharina Eggensperger %E Roman Garnett %E Joaquin Vanschoren %E Marius Lindauer %E Jacob R. Gardner %F pmlr-v256-seng24a %I PMLR %P 10/1--20 %U https://proceedings.mlr.press/v256/seng24a.html %V 256 %X Time series forecasting is ubiquitous in many disciplines. A recent hybrid architecture named predictive Whittle networks (PWNs) tackles this task by employing two distinct modules, a tractable probabilistic model and a neural forecaster, with the former guiding the latter by providing likelihoods about predictions during training. Although PWNs achieve state-of-the-art accuracy, finding the optimal type of probabilistic model and neural forecaster (macro-architecture search) and the architecture of each module (micro-architecture search) of such hybrid models remains difficult and time-consuming. Current one-shot neural architecture search (NAS) methods approach this challenge by focusing on either the micro or the macro aspect, overlooking mutual impact, and could attain the overall optimization only sequentially. To overcome these limitations, we introduce a bi-level one-shot NAS method that optimizes such hybrid architectures simultaneously, leveraging the relationships between the micro and the macro architectural levels. We empirically demonstrate that the hybrid architectures found by our method outperform human-designed and overparameterized ones on various challenging datasets. Furthermore, we unveil insights about underlying connections between architectural choices and temporal features.
APA
Seng, J., Kalter, F., Yu, Z., Ventola, F. & Kersting, K.. (2024). Bi-Level One-Shot Architecture Search for Probabilistic Time Series Forecasting. Proceedings of the Third International Conference on Automated Machine Learning, in Proceedings of Machine Learning Research 256:10/1-20 Available from https://proceedings.mlr.press/v256/seng24a.html.

Related Material