A Statistical Investigation of Long Memory in Language and Music
[edit]
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:23942403, 2019.
Abstract
Representation and learning of longrange dependencies is a central challenge confronted in modern applications of machine learning to sequence data. Yet despite the prominence of this issue, the basic problem of measuring longrange dependence, either in a given data source or as represented in a trained deep model, remains largely limited to heuristic tools. We contribute a statistical framework for investigating longrange dependence in current applications of deep sequence modeling, drawing on the welldeveloped theory of long memory stochastic processes. This framework yields testable implications concerning the relationship between long memory in realworld data and its learned representation in a deep learning architecture, which are explored through a semiparametric framework adapted to the highdimensional setting.
Related Material


