High-Dimensional Structured Feature Screening Using Binary Markov Random Fields

Jie Liu, Chunming Zhang, Catherine Mccarty, Peggy Peissig, Elizabeth Burnside, David Page
; Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:712-721, 2012.

Abstract

Feature screening is a useful feature selection approach for high-dimensional data when the goal is to identify all the features relevant to the response variable. However, common feature screening methods do not take into account the correlation structure of the covariate space. We propose the concept of a feature relevance network, a binary Markov random field to represent the relevance of each individual feature by potentials on the nodes, and represent the correlation structure by potentials on the edges. By performing inference on the feature relevance network, we can accordingly select relevant features. Our algorithm does not yield sparsity, which is different from the particular popular family of feature selection approaches based on penalized least squares or penalized pseudo-likelihood. We give one concrete algorithm under this framework and show its superior performance over common feature selection methods in terms of prediction error and recovery of the truly relevant features on real-world data and synthetic data.

Cite this Paper


BibTeX
@InProceedings{pmlr-v22-liu12b, title = {High-Dimensional Structured Feature Screening Using Binary Markov Random Fields}, author = {Jie Liu and Chunming Zhang and Catherine Mccarty and Peggy Peissig and Elizabeth Burnside and David Page}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {712--721}, year = {2012}, editor = {Neil D. Lawrence and Mark Girolami}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/liu12b/liu12b.pdf}, url = {http://proceedings.mlr.press/v22/liu12b.html}, abstract = {Feature screening is a useful feature selection approach for high-dimensional data when the goal is to identify all the features relevant to the response variable. However, common feature screening methods do not take into account the correlation structure of the covariate space. We propose the concept of a feature relevance network, a binary Markov random field to represent the relevance of each individual feature by potentials on the nodes, and represent the correlation structure by potentials on the edges. By performing inference on the feature relevance network, we can accordingly select relevant features. Our algorithm does not yield sparsity, which is different from the particular popular family of feature selection approaches based on penalized least squares or penalized pseudo-likelihood. We give one concrete algorithm under this framework and show its superior performance over common feature selection methods in terms of prediction error and recovery of the truly relevant features on real-world data and synthetic data.} }
Endnote
%0 Conference Paper %T High-Dimensional Structured Feature Screening Using Binary Markov Random Fields %A Jie Liu %A Chunming Zhang %A Catherine Mccarty %A Peggy Peissig %A Elizabeth Burnside %A David Page %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-liu12b %I PMLR %J Proceedings of Machine Learning Research %P 712--721 %U http://proceedings.mlr.press %V 22 %W PMLR %X Feature screening is a useful feature selection approach for high-dimensional data when the goal is to identify all the features relevant to the response variable. However, common feature screening methods do not take into account the correlation structure of the covariate space. We propose the concept of a feature relevance network, a binary Markov random field to represent the relevance of each individual feature by potentials on the nodes, and represent the correlation structure by potentials on the edges. By performing inference on the feature relevance network, we can accordingly select relevant features. Our algorithm does not yield sparsity, which is different from the particular popular family of feature selection approaches based on penalized least squares or penalized pseudo-likelihood. We give one concrete algorithm under this framework and show its superior performance over common feature selection methods in terms of prediction error and recovery of the truly relevant features on real-world data and synthetic data.
RIS
TY - CPAPER TI - High-Dimensional Structured Feature Screening Using Binary Markov Random Fields AU - Jie Liu AU - Chunming Zhang AU - Catherine Mccarty AU - Peggy Peissig AU - Elizabeth Burnside AU - David Page BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics PY - 2012/03/21 DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-liu12b PB - PMLR SP - 712 DP - PMLR EP - 721 L1 - http://proceedings.mlr.press/v22/liu12b/liu12b.pdf UR - http://proceedings.mlr.press/v22/liu12b.html AB - Feature screening is a useful feature selection approach for high-dimensional data when the goal is to identify all the features relevant to the response variable. However, common feature screening methods do not take into account the correlation structure of the covariate space. We propose the concept of a feature relevance network, a binary Markov random field to represent the relevance of each individual feature by potentials on the nodes, and represent the correlation structure by potentials on the edges. By performing inference on the feature relevance network, we can accordingly select relevant features. Our algorithm does not yield sparsity, which is different from the particular popular family of feature selection approaches based on penalized least squares or penalized pseudo-likelihood. We give one concrete algorithm under this framework and show its superior performance over common feature selection methods in terms of prediction error and recovery of the truly relevant features on real-world data and synthetic data. ER -
APA
Liu, J., Zhang, C., Mccarty, C., Peissig, P., Burnside, E. & Page, D.. (2012). High-Dimensional Structured Feature Screening Using Binary Markov Random Fields. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in PMLR 22:712-721

Related Material