Learning SingleIndex Models in Gaussian Space
[edit]
Proceedings of the 31st Conference On Learning Theory, PMLR 75:18871930, 2018.
Abstract
We consider regression problems where the response is a smooth but nonlinear function of a $k$dimensional projection of $p$ normallydistributed covariates, contaminated with additive Gaussian noise. The goal is to recover the range of the $k$dimensional projection, i.e., the index space. This model is called the multiindex model, and the $k=1$ case is called the singleindex model. For the singleindex model, we characterize the population landscape of a natural semiparametric maximum likelihood objective in terms of the link function and prove that it has no spurious local minima. We also propose and analyze an efficient iterative procedure that recovers the index space up to error $\epsilon$ using a sample size $\tilde{O}(p^{O(R^2/\mu)} + p/\epsilon^2)$, where $R$ and $\mu$, respectively, parameterize the smoothness of the link function and the signal strength. When a multiindex model is incorrectly specified as a singleindex model, we prove that essentially the same procedure, with sample size $\tilde{O}(p^{O(kR^2/\mu)} + p/\epsilon^2)$, returns a vector that is $\epsilon$close to being completely in the index space.
Related Material


