Ultrahigh Dimensional Feature Screening via RKHS Embeddings

Krishnakumar Balasubramanian, Bharath Sriperumbudur, Guy Lebanon
Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, PMLR 31:126-134, 2013.

Abstract

Feature screening is a key step in handling ultrahigh dimensional data sets that are ubiquitous in modern statistical problems. Over the last decade, convex relaxation based approaches (e.g., Lasso/sparse additive model) have been extensively developed and analyzed for feature selection in high dimensional regime. But in the ultrahigh dimensional regime, these approaches suffer from several problems, both computationally and statistically. To overcome these issues, in this paper, we propose a novel Hilbert space embedding based approach to independence screening for ultrahigh dimensional data sets. The proposed approach is model-free (i.e., no model assumption is made between response and predictors) and could handle non-standard (e.g., graphs) and multivariate outputs directly. We establish the sure screening property of the proposed approach in the ultrahigh dimensional regime, and experimentally demonstrate its advantages and superiority over other approaches on several synthetic and real data sets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v31-balasubramanian13a, title = {Ultrahigh Dimensional Feature Screening via RKHS Embeddings}, author = {Balasubramanian, Krishnakumar and Sriperumbudur, Bharath and Lebanon, Guy}, booktitle = {Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics}, pages = {126--134}, year = {2013}, editor = {Carvalho, Carlos M. and Ravikumar, Pradeep}, volume = {31}, series = {Proceedings of Machine Learning Research}, address = {Scottsdale, Arizona, USA}, month = {29 Apr--01 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v31/balasubramanian13a.pdf}, url = {https://proceedings.mlr.press/v31/balasubramanian13a.html}, abstract = {Feature screening is a key step in handling ultrahigh dimensional data sets that are ubiquitous in modern statistical problems. Over the last decade, convex relaxation based approaches (e.g., Lasso/sparse additive model) have been extensively developed and analyzed for feature selection in high dimensional regime. But in the ultrahigh dimensional regime, these approaches suffer from several problems, both computationally and statistically. To overcome these issues, in this paper, we propose a novel Hilbert space embedding based approach to independence screening for ultrahigh dimensional data sets. The proposed approach is model-free (i.e., no model assumption is made between response and predictors) and could handle non-standard (e.g., graphs) and multivariate outputs directly. We establish the sure screening property of the proposed approach in the ultrahigh dimensional regime, and experimentally demonstrate its advantages and superiority over other approaches on several synthetic and real data sets.} }
Endnote
%0 Conference Paper %T Ultrahigh Dimensional Feature Screening via RKHS Embeddings %A Krishnakumar Balasubramanian %A Bharath Sriperumbudur %A Guy Lebanon %B Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2013 %E Carlos M. Carvalho %E Pradeep Ravikumar %F pmlr-v31-balasubramanian13a %I PMLR %P 126--134 %U https://proceedings.mlr.press/v31/balasubramanian13a.html %V 31 %X Feature screening is a key step in handling ultrahigh dimensional data sets that are ubiquitous in modern statistical problems. Over the last decade, convex relaxation based approaches (e.g., Lasso/sparse additive model) have been extensively developed and analyzed for feature selection in high dimensional regime. But in the ultrahigh dimensional regime, these approaches suffer from several problems, both computationally and statistically. To overcome these issues, in this paper, we propose a novel Hilbert space embedding based approach to independence screening for ultrahigh dimensional data sets. The proposed approach is model-free (i.e., no model assumption is made between response and predictors) and could handle non-standard (e.g., graphs) and multivariate outputs directly. We establish the sure screening property of the proposed approach in the ultrahigh dimensional regime, and experimentally demonstrate its advantages and superiority over other approaches on several synthetic and real data sets.
RIS
TY - CPAPER TI - Ultrahigh Dimensional Feature Screening via RKHS Embeddings AU - Krishnakumar Balasubramanian AU - Bharath Sriperumbudur AU - Guy Lebanon BT - Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics DA - 2013/04/29 ED - Carlos M. Carvalho ED - Pradeep Ravikumar ID - pmlr-v31-balasubramanian13a PB - PMLR DP - Proceedings of Machine Learning Research VL - 31 SP - 126 EP - 134 L1 - http://proceedings.mlr.press/v31/balasubramanian13a.pdf UR - https://proceedings.mlr.press/v31/balasubramanian13a.html AB - Feature screening is a key step in handling ultrahigh dimensional data sets that are ubiquitous in modern statistical problems. Over the last decade, convex relaxation based approaches (e.g., Lasso/sparse additive model) have been extensively developed and analyzed for feature selection in high dimensional regime. But in the ultrahigh dimensional regime, these approaches suffer from several problems, both computationally and statistically. To overcome these issues, in this paper, we propose a novel Hilbert space embedding based approach to independence screening for ultrahigh dimensional data sets. The proposed approach is model-free (i.e., no model assumption is made between response and predictors) and could handle non-standard (e.g., graphs) and multivariate outputs directly. We establish the sure screening property of the proposed approach in the ultrahigh dimensional regime, and experimentally demonstrate its advantages and superiority over other approaches on several synthetic and real data sets. ER -
APA
Balasubramanian, K., Sriperumbudur, B. & Lebanon, G.. (2013). Ultrahigh Dimensional Feature Screening via RKHS Embeddings. Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 31:126-134 Available from https://proceedings.mlr.press/v31/balasubramanian13a.html.

Related Material