Random projection design for scalable implicit smoothing of randomly observed stochastic processes


Francois Belletti, Evan Sparks, Alexandre Bayen, Joseph Gonzalez ;
Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR 54:700-708, 2017.


Sampling at random timestamps, long range dependencies, and scale hamper standard meth- ods for multivariate time series analysis. In this paper we present a novel estimator for cross-covariance of randomly observed time series which unravels the dynamics of an unobserved stochastic process. We analyze the statistical properties of our estimator without needing the assumption that observation timestamps are independent from the process of interest and show that our solution is not hindered by the issues affecting standard estimators for cross-covariance. We implement and evaluate our statistically sound and scalable approach in the distributed setting using Apache Spark and demonstrate its ability to unravel causal dynamics on both simulations and high-frequency financial trading data.

Related Material