Extended Deep Adaptive Input Normalization for Preprocessing Time Series Data for Neural Networks

Marcus A. K. September, Francesco Sanna Passino, Leonie Goldmann, Anton Hinel
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:1891-1899, 2024.

Abstract

Data preprocessing is a crucial part of any machine learning pipeline, and it can have a significant impact on both performance and training efficiency. This is especially evident when using deep neural networks for time series prediction and classification: real-world time series data often exhibit irregularities such as multi-modality, skewness and outliers, and the model performance can degrade rapidly if these characteristics are not adequately addressed. In this work, we propose the EDAIN (Extended Deep Adaptive Input Normalization) layer, a novel adaptive neural layer that learns how to appropriately normalize irregular time series data for a given task in an end-to-end fashion, instead of using a fixed normalization scheme. This is achieved by optimizing its unknown parameters simultaneously with the deep neural network using back-propagation. Our experiments, conducted using synthetic data, a credit default prediction dataset, and a large-scale limit order book benchmark dataset, demonstrate the superior performance of the EDAIN layer when compared to conventional normalization methods and existing adaptive time series preprocessing layers.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-september24a, title = {Extended Deep Adaptive Input Normalization for Preprocessing Time Series Data for Neural Networks}, author = {September, Marcus A. K. and Sanna Passino, Francesco and Goldmann, Leonie and Hinel, Anton}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {1891--1899}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/september24a/september24a.pdf}, url = {https://proceedings.mlr.press/v238/september24a.html}, abstract = {Data preprocessing is a crucial part of any machine learning pipeline, and it can have a significant impact on both performance and training efficiency. This is especially evident when using deep neural networks for time series prediction and classification: real-world time series data often exhibit irregularities such as multi-modality, skewness and outliers, and the model performance can degrade rapidly if these characteristics are not adequately addressed. In this work, we propose the EDAIN (Extended Deep Adaptive Input Normalization) layer, a novel adaptive neural layer that learns how to appropriately normalize irregular time series data for a given task in an end-to-end fashion, instead of using a fixed normalization scheme. This is achieved by optimizing its unknown parameters simultaneously with the deep neural network using back-propagation. Our experiments, conducted using synthetic data, a credit default prediction dataset, and a large-scale limit order book benchmark dataset, demonstrate the superior performance of the EDAIN layer when compared to conventional normalization methods and existing adaptive time series preprocessing layers.} }
Endnote
%0 Conference Paper %T Extended Deep Adaptive Input Normalization for Preprocessing Time Series Data for Neural Networks %A Marcus A. K. September %A Francesco Sanna Passino %A Leonie Goldmann %A Anton Hinel %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-september24a %I PMLR %P 1891--1899 %U https://proceedings.mlr.press/v238/september24a.html %V 238 %X Data preprocessing is a crucial part of any machine learning pipeline, and it can have a significant impact on both performance and training efficiency. This is especially evident when using deep neural networks for time series prediction and classification: real-world time series data often exhibit irregularities such as multi-modality, skewness and outliers, and the model performance can degrade rapidly if these characteristics are not adequately addressed. In this work, we propose the EDAIN (Extended Deep Adaptive Input Normalization) layer, a novel adaptive neural layer that learns how to appropriately normalize irregular time series data for a given task in an end-to-end fashion, instead of using a fixed normalization scheme. This is achieved by optimizing its unknown parameters simultaneously with the deep neural network using back-propagation. Our experiments, conducted using synthetic data, a credit default prediction dataset, and a large-scale limit order book benchmark dataset, demonstrate the superior performance of the EDAIN layer when compared to conventional normalization methods and existing adaptive time series preprocessing layers.
APA
September, M.A.K., Sanna Passino, F., Goldmann, L. & Hinel, A.. (2024). Extended Deep Adaptive Input Normalization for Preprocessing Time Series Data for Neural Networks. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:1891-1899 Available from https://proceedings.mlr.press/v238/september24a.html.

Related Material