Sparse Multivariate Bernoulli Processes in High Dimensions
[edit]
Proceedings of Machine Learning Research, PMLR 89:457466, 2019.
Abstract
We consider the problem of estimating the parameters of a multivariate Bernoulli process with autoregressive feedback in the highdimensional setting where the number of samples available is much less than the number of parameters. This problem arises in learning interconnections of networks of dynamical systems with spiking or binary valued data. We also allow the process to depend on its past up to a lag p, for a general $p \geq 1$, allowing for more realistic modeling in many applications. We propose and analyze an $\ell_1$regularized maximum likelihood (ML) estimator under the assumption that the parameter tensor is approximately sparse. Rigorous analysis of such estimators is made challenging by the dependent and nonGaussian nature of the process as well as the presence of the nonlinearities and multilevel feedback. We derive precise upper bounds on the meansquared estimation error in terms of the number of samples, dimensions of the process, the lag $p$ and other key statistical properties of the model. The ideas presented can be used in the rigorous highdimensional analysis of regularized $M$estimators for other sparse nonlinear and nonGaussian processes with longrange dependence.
Related Material


