Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting

Kashif Rasul, Calvin Seward, Ingmar Schuster, Roland Vollgraf
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:8857-8868, 2021.

Abstract

In this work, we propose TimeGrad, an autoregressive model for multivariate probabilistic time series forecasting which samples from the data distribution at each time step by estimating its gradient. To this end, we use diffusion probabilistic models, a class of latent variable models closely connected to score matching and energy-based methods. Our model learns gradients by optimizing a variational bound on the data likelihood and at inference time converts white noise into a sample of the distribution of interest through a Markov chain using Langevin sampling. We demonstrate experimentally that the proposed autoregressive denoising diffusion model is the new state-of-the-art multivariate probabilistic forecasting method on real-world data sets with thousands of correlated dimensions. We hope that this method is a useful tool for practitioners and lays the foundation for future research in this area.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-rasul21a, title = {Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting}, author = {Rasul, Kashif and Seward, Calvin and Schuster, Ingmar and Vollgraf, Roland}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {8857--8868}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/rasul21a/rasul21a.pdf}, url = {https://proceedings.mlr.press/v139/rasul21a.html}, abstract = {In this work, we propose TimeGrad, an autoregressive model for multivariate probabilistic time series forecasting which samples from the data distribution at each time step by estimating its gradient. To this end, we use diffusion probabilistic models, a class of latent variable models closely connected to score matching and energy-based methods. Our model learns gradients by optimizing a variational bound on the data likelihood and at inference time converts white noise into a sample of the distribution of interest through a Markov chain using Langevin sampling. We demonstrate experimentally that the proposed autoregressive denoising diffusion model is the new state-of-the-art multivariate probabilistic forecasting method on real-world data sets with thousands of correlated dimensions. We hope that this method is a useful tool for practitioners and lays the foundation for future research in this area.} }
Endnote
%0 Conference Paper %T Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting %A Kashif Rasul %A Calvin Seward %A Ingmar Schuster %A Roland Vollgraf %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-rasul21a %I PMLR %P 8857--8868 %U https://proceedings.mlr.press/v139/rasul21a.html %V 139 %X In this work, we propose TimeGrad, an autoregressive model for multivariate probabilistic time series forecasting which samples from the data distribution at each time step by estimating its gradient. To this end, we use diffusion probabilistic models, a class of latent variable models closely connected to score matching and energy-based methods. Our model learns gradients by optimizing a variational bound on the data likelihood and at inference time converts white noise into a sample of the distribution of interest through a Markov chain using Langevin sampling. We demonstrate experimentally that the proposed autoregressive denoising diffusion model is the new state-of-the-art multivariate probabilistic forecasting method on real-world data sets with thousands of correlated dimensions. We hope that this method is a useful tool for practitioners and lays the foundation for future research in this area.
APA
Rasul, K., Seward, C., Schuster, I. & Vollgraf, R.. (2021). Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:8857-8868 Available from https://proceedings.mlr.press/v139/rasul21a.html.

Related Material