Utilizing Expert Features for Contrastive Learning of Time-Series Representations

Manuel T Nonnenmacher, Lukas Oldenburg, Ingo Steinwart, David Reeb
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:16969-16989, 2022.

Abstract

We present an approach that incorporates expert knowledge for time-series representation learning. Our method employs expert features to replace the commonly used data transformations in previous contrastive learning approaches. We do this since time-series data frequently stems from the industrial or medical field where expert features are often available from domain experts, while transformations are generally elusive for time-series data. We start by proposing two properties that useful time-series representations should fulfill and show that current representation learning approaches do not ensure these properties. We therefore devise ExpCLR, a novel contrastive learning approach built on an objective that utilizes expert features to encourage both properties for the learned representation. Finally, we demonstrate on three real-world time-series datasets that ExpCLR surpasses several state-of-the-art methods for both unsupervised and semi-supervised representation learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-nonnenmacher22a, title = {Utilizing Expert Features for Contrastive Learning of Time-Series Representations}, author = {Nonnenmacher, Manuel T and Oldenburg, Lukas and Steinwart, Ingo and Reeb, David}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {16969--16989}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/nonnenmacher22a/nonnenmacher22a.pdf}, url = {https://proceedings.mlr.press/v162/nonnenmacher22a.html}, abstract = {We present an approach that incorporates expert knowledge for time-series representation learning. Our method employs expert features to replace the commonly used data transformations in previous contrastive learning approaches. We do this since time-series data frequently stems from the industrial or medical field where expert features are often available from domain experts, while transformations are generally elusive for time-series data. We start by proposing two properties that useful time-series representations should fulfill and show that current representation learning approaches do not ensure these properties. We therefore devise ExpCLR, a novel contrastive learning approach built on an objective that utilizes expert features to encourage both properties for the learned representation. Finally, we demonstrate on three real-world time-series datasets that ExpCLR surpasses several state-of-the-art methods for both unsupervised and semi-supervised representation learning.} }
Endnote
%0 Conference Paper %T Utilizing Expert Features for Contrastive Learning of Time-Series Representations %A Manuel T Nonnenmacher %A Lukas Oldenburg %A Ingo Steinwart %A David Reeb %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-nonnenmacher22a %I PMLR %P 16969--16989 %U https://proceedings.mlr.press/v162/nonnenmacher22a.html %V 162 %X We present an approach that incorporates expert knowledge for time-series representation learning. Our method employs expert features to replace the commonly used data transformations in previous contrastive learning approaches. We do this since time-series data frequently stems from the industrial or medical field where expert features are often available from domain experts, while transformations are generally elusive for time-series data. We start by proposing two properties that useful time-series representations should fulfill and show that current representation learning approaches do not ensure these properties. We therefore devise ExpCLR, a novel contrastive learning approach built on an objective that utilizes expert features to encourage both properties for the learned representation. Finally, we demonstrate on three real-world time-series datasets that ExpCLR surpasses several state-of-the-art methods for both unsupervised and semi-supervised representation learning.
APA
Nonnenmacher, M.T., Oldenburg, L., Steinwart, I. & Reeb, D.. (2022). Utilizing Expert Features for Contrastive Learning of Time-Series Representations. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:16969-16989 Available from https://proceedings.mlr.press/v162/nonnenmacher22a.html.

Related Material