High-dimensional Non-Gaussian Single Index Models via Thresholded Score Function Estimation

[edit]

Zhuoran Yang, Krishnakumar Balasubramanian, Han Liu ;
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:3851-3860, 2017.

Abstract

We consider estimating the parametric component of single index models in high dimensions. Compared with existing work, we do not require the covariate to be normally distributed. Utilizing Stein’s Lemma, we propose estimators based on the score function of the covariate. Moreover, to handle score function and response variables that are heavy-tailed, our estimators are constructed via carefully thresholding their empirical counterparts. Under a bounded fourth moment condition, we establish optimal statistical rates of convergence for the proposed estimators. Extensive numerical experiments are provided to back up our theory.

Related Material