A Computationally Efficient Method for Estimating Semi Parametric Regression Functions

Xia Cui, Ying Lu, Heng Peng
Proceedings of the 1st International Workshop on Feature Extraction: Modern Questions and Challenges at NIPS 2015, PMLR 44:90-102, 2015.

Abstract

Bias reduction is an important condition for effective feature extraction. Utilizing recent theoretical results in high dimensional statistical modeling, we propose a model-free yet computationally simple approach to estimate the partially linear model Y=Xβ+g(Z)+\varepsilon. Based on partitioning the support of Z, a simple local average is used to approximate the response surface g(Z). The model can be estimated via least squares and no tuning parameter is needed. The proposed method seeks to strike a balance between computation burden and efficiency of the estimators while minimizing model bias. The desired theoretical properties of the proposed estimators are established. Moreover, since the proposed method bypasses data-driven bandwith selection of traditional nonparametric methods, it avoids the further efficiency loss due to computation burden.

Cite this Paper

BibTeX
@InProceedings{pmlr-v44-CuiLuPeng15, title = {A Computationally Efficient Method for Estimating Semi Parametric Regression Functions}, author = {Cui, Xia and Lu, Ying and Peng, Heng}, booktitle = {Proceedings of the 1st International Workshop on Feature Extraction: Modern Questions and Challenges at NIPS 2015}, pages = {90--102}, year = {2015}, editor = {Storcheus, Dmitry and Rostamizadeh, Afshin and Kumar, Sanjiv}, volume = {44}, series = {Proceedings of Machine Learning Research}, address = {Montreal, Canada}, month = {11 Dec}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v44/CuiLuPeng15.pdf}, url = {https://proceedings.mlr.press/v44/CuiLuPeng15.html}, abstract = {Bias reduction is an important condition for effective feature extraction. Utilizing recent theoretical results in high dimensional statistical modeling, we propose a model-free yet computationally simple approach to estimate the partially linear model Y=Xβ+g(Z)+\varepsilon. Based on partitioning the support of Z, a simple local average is used to approximate the response surface g(Z). The model can be estimated via least squares and no tuning parameter is needed. The proposed method seeks to strike a balance between computation burden and efficiency of the estimators while minimizing model bias. The desired theoretical properties of the proposed estimators are established. Moreover, since the proposed method bypasses data-driven bandwith selection of traditional nonparametric methods, it avoids the further efficiency loss due to computation burden.} }
Endnote
%0 Conference Paper %T A Computationally Efficient Method for Estimating Semi Parametric Regression Functions %A Xia Cui %A Ying Lu %A Heng Peng %B Proceedings of the 1st International Workshop on Feature Extraction: Modern Questions and Challenges at NIPS 2015 %C Proceedings of Machine Learning Research %D 2015 %E Dmitry Storcheus %E Afshin Rostamizadeh %E Sanjiv Kumar %F pmlr-v44-CuiLuPeng15 %I PMLR %P 90--102 %U https://proceedings.mlr.press/v44/CuiLuPeng15.html %V 44 %X Bias reduction is an important condition for effective feature extraction. Utilizing recent theoretical results in high dimensional statistical modeling, we propose a model-free yet computationally simple approach to estimate the partially linear model Y=Xβ+g(Z)+\varepsilon. Based on partitioning the support of Z, a simple local average is used to approximate the response surface g(Z). The model can be estimated via least squares and no tuning parameter is needed. The proposed method seeks to strike a balance between computation burden and efficiency of the estimators while minimizing model bias. The desired theoretical properties of the proposed estimators are established. Moreover, since the proposed method bypasses data-driven bandwith selection of traditional nonparametric methods, it avoids the further efficiency loss due to computation burden.
RIS
TY - CPAPER TI - A Computationally Efficient Method for Estimating Semi Parametric Regression Functions AU - Xia Cui AU - Ying Lu AU - Heng Peng BT - Proceedings of the 1st International Workshop on Feature Extraction: Modern Questions and Challenges at NIPS 2015 DA - 2015/12/08 ED - Dmitry Storcheus ED - Afshin Rostamizadeh ED - Sanjiv Kumar ID - pmlr-v44-CuiLuPeng15 PB - PMLR DP - Proceedings of Machine Learning Research VL - 44 SP - 90 EP - 102 L1 - http://proceedings.mlr.press/v44/CuiLuPeng15.pdf UR - https://proceedings.mlr.press/v44/CuiLuPeng15.html AB - Bias reduction is an important condition for effective feature extraction. Utilizing recent theoretical results in high dimensional statistical modeling, we propose a model-free yet computationally simple approach to estimate the partially linear model Y=Xβ+g(Z)+\varepsilon. Based on partitioning the support of Z, a simple local average is used to approximate the response surface g(Z). The model can be estimated via least squares and no tuning parameter is needed. The proposed method seeks to strike a balance between computation burden and efficiency of the estimators while minimizing model bias. The desired theoretical properties of the proposed estimators are established. Moreover, since the proposed method bypasses data-driven bandwith selection of traditional nonparametric methods, it avoids the further efficiency loss due to computation burden. ER -
APA
Cui, X., Lu, Y. & Peng, H.. (2015). A Computationally Efficient Method for Estimating Semi Parametric Regression Functions. Proceedings of the 1st International Workshop on Feature Extraction: Modern Questions and Challenges at NIPS 2015, in Proceedings of Machine Learning Research 44:90-102 Available from https://proceedings.mlr.press/v44/CuiLuPeng15.html.

Related Material