Approximation Theory of Convolutional Architectures for Time Series Modelling

Haotian Jiang, Zhong Li, Qianxiao Li
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:4961-4970, 2021.

Abstract

We study the approximation properties of convolutional architectures applied to time series modelling, which can be formulated mathematically as a functional approximation problem. In the recurrent setting, recent results reveal an intricate connection between approximation efficiency and memory structures in the data generation process. In this paper, we derive parallel results for convolutional architectures, with WaveNet being a prime example. Our results reveal that in this new setting, approximation efficiency is not only characterised by memory, but also additional fine structures in the target relationship. This leads to a novel definition of spectrum-based regularity that measures the complexity of temporal relationships under the convolutional approximation scheme. These analyses provide a foundation to understand the differences between architectural choices for time series modelling and can give theoretically grounded guidance for practical applications.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-jiang21d, title = {Approximation Theory of Convolutional Architectures for Time Series Modelling}, author = {Jiang, Haotian and Li, Zhong and Li, Qianxiao}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {4961--4970}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/jiang21d/jiang21d.pdf}, url = {https://proceedings.mlr.press/v139/jiang21d.html}, abstract = {We study the approximation properties of convolutional architectures applied to time series modelling, which can be formulated mathematically as a functional approximation problem. In the recurrent setting, recent results reveal an intricate connection between approximation efficiency and memory structures in the data generation process. In this paper, we derive parallel results for convolutional architectures, with WaveNet being a prime example. Our results reveal that in this new setting, approximation efficiency is not only characterised by memory, but also additional fine structures in the target relationship. This leads to a novel definition of spectrum-based regularity that measures the complexity of temporal relationships under the convolutional approximation scheme. These analyses provide a foundation to understand the differences between architectural choices for time series modelling and can give theoretically grounded guidance for practical applications.} }
Endnote
%0 Conference Paper %T Approximation Theory of Convolutional Architectures for Time Series Modelling %A Haotian Jiang %A Zhong Li %A Qianxiao Li %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-jiang21d %I PMLR %P 4961--4970 %U https://proceedings.mlr.press/v139/jiang21d.html %V 139 %X We study the approximation properties of convolutional architectures applied to time series modelling, which can be formulated mathematically as a functional approximation problem. In the recurrent setting, recent results reveal an intricate connection between approximation efficiency and memory structures in the data generation process. In this paper, we derive parallel results for convolutional architectures, with WaveNet being a prime example. Our results reveal that in this new setting, approximation efficiency is not only characterised by memory, but also additional fine structures in the target relationship. This leads to a novel definition of spectrum-based regularity that measures the complexity of temporal relationships under the convolutional approximation scheme. These analyses provide a foundation to understand the differences between architectural choices for time series modelling and can give theoretically grounded guidance for practical applications.
APA
Jiang, H., Li, Z. & Li, Q.. (2021). Approximation Theory of Convolutional Architectures for Time Series Modelling. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:4961-4970 Available from https://proceedings.mlr.press/v139/jiang21d.html.

Related Material