Vector Quantized Time Series Generation with a Bidirectional Prior Model

Daesoo Lee, Sara Malacarne, Erlend Aune
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:7665-7693, 2023.

Abstract

Time series generation (TSG) studies have mainly focused on the use of Generative Adversarial Networks (GANs) combined with recurrent neural network (RNN) variants. However, the fundamental limitations and challenges of training GANs still remain. In addition, the RNN-family typically has difficulties with temporal consistency between distant timesteps. Motivated by the successes in the image generation (IMG) domain, we propose TimeVQVAE, the first work, to our knowledge, that uses vector quantization (VQ) techniques to address the TSG problem. Moreover, the priors of the discrete latent spaces are learned with bidirectional transformer models that can better capture global temporal consistency. We also propose VQ modeling in a time-frequency domain, separated into low-frequency (LF) and high-frequency (HF). This allows us to retain important characteristics of the time series and, in turn, generate new synthetic signals that are of better quality, with sharper changes in modularity, than its competing TSG methods. Our experimental evaluation is conducted on all datasets from the UCR archive, using well-established metrics in the IMG literature, such as Frechet inception distance and inception scores. Our implementation on GitHub: https://github.com/ML4ITS/TimeVQVAE.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-lee23d, title = {Vector Quantized Time Series Generation with a Bidirectional Prior Model}, author = {Lee, Daesoo and Malacarne, Sara and Aune, Erlend}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {7665--7693}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/lee23d/lee23d.pdf}, url = {https://proceedings.mlr.press/v206/lee23d.html}, abstract = {Time series generation (TSG) studies have mainly focused on the use of Generative Adversarial Networks (GANs) combined with recurrent neural network (RNN) variants. However, the fundamental limitations and challenges of training GANs still remain. In addition, the RNN-family typically has difficulties with temporal consistency between distant timesteps. Motivated by the successes in the image generation (IMG) domain, we propose TimeVQVAE, the first work, to our knowledge, that uses vector quantization (VQ) techniques to address the TSG problem. Moreover, the priors of the discrete latent spaces are learned with bidirectional transformer models that can better capture global temporal consistency. We also propose VQ modeling in a time-frequency domain, separated into low-frequency (LF) and high-frequency (HF). This allows us to retain important characteristics of the time series and, in turn, generate new synthetic signals that are of better quality, with sharper changes in modularity, than its competing TSG methods. Our experimental evaluation is conducted on all datasets from the UCR archive, using well-established metrics in the IMG literature, such as Frechet inception distance and inception scores. Our implementation on GitHub: https://github.com/ML4ITS/TimeVQVAE.} }
Endnote
%0 Conference Paper %T Vector Quantized Time Series Generation with a Bidirectional Prior Model %A Daesoo Lee %A Sara Malacarne %A Erlend Aune %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-lee23d %I PMLR %P 7665--7693 %U https://proceedings.mlr.press/v206/lee23d.html %V 206 %X Time series generation (TSG) studies have mainly focused on the use of Generative Adversarial Networks (GANs) combined with recurrent neural network (RNN) variants. However, the fundamental limitations and challenges of training GANs still remain. In addition, the RNN-family typically has difficulties with temporal consistency between distant timesteps. Motivated by the successes in the image generation (IMG) domain, we propose TimeVQVAE, the first work, to our knowledge, that uses vector quantization (VQ) techniques to address the TSG problem. Moreover, the priors of the discrete latent spaces are learned with bidirectional transformer models that can better capture global temporal consistency. We also propose VQ modeling in a time-frequency domain, separated into low-frequency (LF) and high-frequency (HF). This allows us to retain important characteristics of the time series and, in turn, generate new synthetic signals that are of better quality, with sharper changes in modularity, than its competing TSG methods. Our experimental evaluation is conducted on all datasets from the UCR archive, using well-established metrics in the IMG literature, such as Frechet inception distance and inception scores. Our implementation on GitHub: https://github.com/ML4ITS/TimeVQVAE.
APA
Lee, D., Malacarne, S. & Aune, E.. (2023). Vector Quantized Time Series Generation with a Bidirectional Prior Model. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:7665-7693 Available from https://proceedings.mlr.press/v206/lee23d.html.

Related Material