High-Dimensional Structured Feature Screening Using Binary Markov Random Fields

Jie Liu, Chunming Zhang, Catherine Mccarty, Peggy Peissig, Elizabeth Burnside, David Page
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:712-721, 2012.

Abstract

Feature screening is a useful feature selection approach for high-dimensional data when the goal is to identify all the features relevant to the response variable. However, common feature screening methods do not take into account the correlation structure of the covariate space. We propose the concept of a feature relevance network, a binary Markov random field to represent the relevance of each individual feature by potentials on the nodes, and represent the correlation structure by potentials on the edges. By performing inference on the feature relevance network, we can accordingly select relevant features. Our algorithm does not yield sparsity, which is different from the particular popular family of feature selection approaches based on penalized least squares or penalized pseudo-likelihood. We give one concrete algorithm under this framework and show its superior performance over common feature selection methods in terms of prediction error and recovery of the truly relevant features on real-world data and synthetic data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-liu12b, title = {High-Dimensional Structured Feature Screening Using Binary Markov Random Fields}, author = {Liu, Jie and Zhang, Chunming and Mccarty, Catherine and Peissig, Peggy and Burnside, Elizabeth and Page, David}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {712--721}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/liu12b/liu12b.pdf}, url = {https://proceedings.mlr.press/v22/liu12b.html}, abstract = {Feature screening is a useful feature selection approach for high-dimensional data when the goal is to identify all the features relevant to the response variable. However, common feature screening methods do not take into account the correlation structure of the covariate space. We propose the concept of a feature relevance network, a binary Markov random field to represent the relevance of each individual feature by potentials on the nodes, and represent the correlation structure by potentials on the edges. By performing inference on the feature relevance network, we can accordingly select relevant features. Our algorithm does not yield sparsity, which is different from the particular popular family of feature selection approaches based on penalized least squares or penalized pseudo-likelihood. We give one concrete algorithm under this framework and show its superior performance over common feature selection methods in terms of prediction error and recovery of the truly relevant features on real-world data and synthetic data.} }
Endnote
%0 Conference Paper %T High-Dimensional Structured Feature Screening Using Binary Markov Random Fields %A Jie Liu %A Chunming Zhang %A Catherine Mccarty %A Peggy Peissig %A Elizabeth Burnside %A David Page %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-liu12b %I PMLR %P 712--721 %U https://proceedings.mlr.press/v22/liu12b.html %V 22 %X Feature screening is a useful feature selection approach for high-dimensional data when the goal is to identify all the features relevant to the response variable. However, common feature screening methods do not take into account the correlation structure of the covariate space. We propose the concept of a feature relevance network, a binary Markov random field to represent the relevance of each individual feature by potentials on the nodes, and represent the correlation structure by potentials on the edges. By performing inference on the feature relevance network, we can accordingly select relevant features. Our algorithm does not yield sparsity, which is different from the particular popular family of feature selection approaches based on penalized least squares or penalized pseudo-likelihood. We give one concrete algorithm under this framework and show its superior performance over common feature selection methods in terms of prediction error and recovery of the truly relevant features on real-world data and synthetic data.
RIS
TY - CPAPER TI - High-Dimensional Structured Feature Screening Using Binary Markov Random Fields AU - Jie Liu AU - Chunming Zhang AU - Catherine Mccarty AU - Peggy Peissig AU - Elizabeth Burnside AU - David Page BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-liu12b PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 712 EP - 721 L1 - http://proceedings.mlr.press/v22/liu12b/liu12b.pdf UR - https://proceedings.mlr.press/v22/liu12b.html AB - Feature screening is a useful feature selection approach for high-dimensional data when the goal is to identify all the features relevant to the response variable. However, common feature screening methods do not take into account the correlation structure of the covariate space. We propose the concept of a feature relevance network, a binary Markov random field to represent the relevance of each individual feature by potentials on the nodes, and represent the correlation structure by potentials on the edges. By performing inference on the feature relevance network, we can accordingly select relevant features. Our algorithm does not yield sparsity, which is different from the particular popular family of feature selection approaches based on penalized least squares or penalized pseudo-likelihood. We give one concrete algorithm under this framework and show its superior performance over common feature selection methods in terms of prediction error and recovery of the truly relevant features on real-world data and synthetic data. ER -
APA
Liu, J., Zhang, C., Mccarty, C., Peissig, P., Burnside, E. & Page, D.. (2012). High-Dimensional Structured Feature Screening Using Binary Markov Random Fields. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:712-721 Available from https://proceedings.mlr.press/v22/liu12b.html.

Related Material