A Consistent Estimator of the Expected Gradient Outerproduct

Shubhendu Trivedi Toyota Technological Institute, Jialei Wang, Samory Kpotufe TTI-Chicago, Gregory Shakhnarovich TT-Chicago
Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, PMLR R12:872-881, 2014.

Abstract

In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely EX $\nabla$f(X) \cdot $\nabla$f(X)$\top$ , is known to recover those directions v $\in$Rd most relevant to predicting the output Y . However, just as in gradient estimation, opti- mal estimators of the EGOP can be expensive in practice. We show that a simple rough estima- tor, much cheaper in practice, suffices to obtain significant improvements on real-world nonpara- metric classification and regression tasks. Fur- thermore, we prove that, despite its simplicity, this rough estimator remains statistically consis- tent under mild conditions.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR12-institute14a, title = {A Consistent Estimator of the Expected Gradient Outerproduct}, author = {Institute, Shubhendu Trivedi Toyota Technological and Wang, Jialei and TTI-Chicago, Samory Kpotufe and TT-Chicago, Gregory Shakhnarovich}, booktitle = {Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence}, pages = {872--881}, year = {2014}, editor = {Zhang, Nevin L. and Tian, Jin}, volume = {R12}, series = {Proceedings of Machine Learning Research}, month = {23--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r12/main/assets/institute14a/institute14a.pdf}, url = {https://proceedings.mlr.press/r12/institute14a.html}, abstract = {In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely EX $\nabla$f(X) \cdot $\nabla$f(X)$\top$ , is known to recover those directions v $\in$Rd most relevant to predicting the output Y . However, just as in gradient estimation, opti- mal estimators of the EGOP can be expensive in practice. We show that a simple rough estima- tor, much cheaper in practice, suffices to obtain significant improvements on real-world nonpara- metric classification and regression tasks. Fur- thermore, we prove that, despite its simplicity, this rough estimator remains statistically consis- tent under mild conditions.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T A Consistent Estimator of the Expected Gradient Outerproduct %A Shubhendu Trivedi Toyota Technological Institute %A Jialei Wang %A Samory Kpotufe TTI-Chicago %A Gregory Shakhnarovich TT-Chicago %B Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2014 %E Nevin L. Zhang %E Jin Tian %F pmlr-vR12-institute14a %I PMLR %P 872--881 %U https://proceedings.mlr.press/r12/institute14a.html %V R12 %X In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely EX $\nabla$f(X) \cdot $\nabla$f(X)$\top$ , is known to recover those directions v $\in$Rd most relevant to predicting the output Y . However, just as in gradient estimation, opti- mal estimators of the EGOP can be expensive in practice. We show that a simple rough estima- tor, much cheaper in practice, suffices to obtain significant improvements on real-world nonpara- metric classification and regression tasks. Fur- thermore, we prove that, despite its simplicity, this rough estimator remains statistically consis- tent under mild conditions. %Z Reissued by PMLR on 04 October 2026.
APA
Institute, S.T.T.T., Wang, J., TTI-Chicago, S.K. & TT-Chicago, G.S.. (2014). A Consistent Estimator of the Expected Gradient Outerproduct. Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R12:872-881 Available from https://proceedings.mlr.press/r12/institute14a.html. Reissued by PMLR on 04 October 2026.

Related Material