A Statistical Investigation of Long Memory in Language and Music

Alexander Greaves-Tunnell, Zaid Harchaoui
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:2394-2403, 2019.

Abstract

Representation and learning of long-range dependencies is a central challenge confronted in modern applications of machine learning to sequence data. Yet despite the prominence of this issue, the basic problem of measuring long-range dependence, either in a given data source or as represented in a trained deep model, remains largely limited to heuristic tools. We contribute a statistical framework for investigating long-range dependence in current applications of deep sequence modeling, drawing on the well-developed theory of long memory stochastic processes. This framework yields testable implications concerning the relationship between long memory in real-world data and its learned representation in a deep learning architecture, which are explored through a semiparametric framework adapted to the high-dimensional setting.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-greaves-tunnell19a, title = {A Statistical Investigation of Long Memory in Language and Music}, author = {Greaves-Tunnell, Alexander and Harchaoui, Zaid}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {2394--2403}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/greaves-tunnell19a/greaves-tunnell19a.pdf}, url = {https://proceedings.mlr.press/v97/greaves-tunnell19a.html}, abstract = {Representation and learning of long-range dependencies is a central challenge confronted in modern applications of machine learning to sequence data. Yet despite the prominence of this issue, the basic problem of measuring long-range dependence, either in a given data source or as represented in a trained deep model, remains largely limited to heuristic tools. We contribute a statistical framework for investigating long-range dependence in current applications of deep sequence modeling, drawing on the well-developed theory of long memory stochastic processes. This framework yields testable implications concerning the relationship between long memory in real-world data and its learned representation in a deep learning architecture, which are explored through a semiparametric framework adapted to the high-dimensional setting.} }
Endnote
%0 Conference Paper %T A Statistical Investigation of Long Memory in Language and Music %A Alexander Greaves-Tunnell %A Zaid Harchaoui %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-greaves-tunnell19a %I PMLR %P 2394--2403 %U https://proceedings.mlr.press/v97/greaves-tunnell19a.html %V 97 %X Representation and learning of long-range dependencies is a central challenge confronted in modern applications of machine learning to sequence data. Yet despite the prominence of this issue, the basic problem of measuring long-range dependence, either in a given data source or as represented in a trained deep model, remains largely limited to heuristic tools. We contribute a statistical framework for investigating long-range dependence in current applications of deep sequence modeling, drawing on the well-developed theory of long memory stochastic processes. This framework yields testable implications concerning the relationship between long memory in real-world data and its learned representation in a deep learning architecture, which are explored through a semiparametric framework adapted to the high-dimensional setting.
APA
Greaves-Tunnell, A. & Harchaoui, Z.. (2019). A Statistical Investigation of Long Memory in Language and Music. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:2394-2403 Available from https://proceedings.mlr.press/v97/greaves-tunnell19a.html.

Related Material