Determinantal Regularization for Ensemble Variable Selection

Veronika Rockova, Gemma Moran, Edward George
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, PMLR 51:1105-1113, 2016.

Abstract

Recent years have seen growing interest in deterministic search approaches to spike-and-slab Bayesian variable selection. Such methods have focused on the goal of finding a global mode to identify a “best model”. However, the report of a single model will be a misleading reflection of the model uncertainty inherent in a highly multimodal posterior. Motivated by non-parametric variational Bayes strategies, we move beyond this limitation by proposing an ensemble optimization approach to identify a collection of representative posterior modes. By deploying determinantal penalty functions as diversity regularizers, our approach performs regularization over multiple locations of the posterior. The key driver of these determinantal penalties is a kernel function that induces repulsion in the latent model space domain.

Cite this Paper

BibTeX
@InProceedings{pmlr-v51-rockova16, title = {Determinantal Regularization for Ensemble Variable Selection}, author = {Rockova, Veronika and Moran, Gemma and George, Edward}, booktitle = {Proceedings of the 19th International Conference on Artificial Intelligence and Statistics}, pages = {1105--1113}, year = {2016}, editor = {Gretton, Arthur and Robert, Christian C.}, volume = {51}, series = {Proceedings of Machine Learning Research}, address = {Cadiz, Spain}, month = {09--11 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v51/rockova16.pdf}, url = {https://proceedings.mlr.press/v51/rockova16.html}, abstract = {Recent years have seen growing interest in deterministic search approaches to spike-and-slab Bayesian variable selection. Such methods have focused on the goal of finding a global mode to identify a “best model”. However, the report of a single model will be a misleading reflection of the model uncertainty inherent in a highly multimodal posterior. Motivated by non-parametric variational Bayes strategies, we move beyond this limitation by proposing an ensemble optimization approach to identify a collection of representative posterior modes. By deploying determinantal penalty functions as diversity regularizers, our approach performs regularization over multiple locations of the posterior. The key driver of these determinantal penalties is a kernel function that induces repulsion in the latent model space domain.} }
Endnote
%0 Conference Paper %T Determinantal Regularization for Ensemble Variable Selection %A Veronika Rockova %A Gemma Moran %A Edward George %B Proceedings of the 19th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2016 %E Arthur Gretton %E Christian C. Robert %F pmlr-v51-rockova16 %I PMLR %P 1105--1113 %U https://proceedings.mlr.press/v51/rockova16.html %V 51 %X Recent years have seen growing interest in deterministic search approaches to spike-and-slab Bayesian variable selection. Such methods have focused on the goal of finding a global mode to identify a “best model”. However, the report of a single model will be a misleading reflection of the model uncertainty inherent in a highly multimodal posterior. Motivated by non-parametric variational Bayes strategies, we move beyond this limitation by proposing an ensemble optimization approach to identify a collection of representative posterior modes. By deploying determinantal penalty functions as diversity regularizers, our approach performs regularization over multiple locations of the posterior. The key driver of these determinantal penalties is a kernel function that induces repulsion in the latent model space domain.
RIS
TY - CPAPER TI - Determinantal Regularization for Ensemble Variable Selection AU - Veronika Rockova AU - Gemma Moran AU - Edward George BT - Proceedings of the 19th International Conference on Artificial Intelligence and Statistics DA - 2016/05/02 ED - Arthur Gretton ED - Christian C. Robert ID - pmlr-v51-rockova16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 51 SP - 1105 EP - 1113 L1 - http://proceedings.mlr.press/v51/rockova16.pdf UR - https://proceedings.mlr.press/v51/rockova16.html AB - Recent years have seen growing interest in deterministic search approaches to spike-and-slab Bayesian variable selection. Such methods have focused on the goal of finding a global mode to identify a “best model”. However, the report of a single model will be a misleading reflection of the model uncertainty inherent in a highly multimodal posterior. Motivated by non-parametric variational Bayes strategies, we move beyond this limitation by proposing an ensemble optimization approach to identify a collection of representative posterior modes. By deploying determinantal penalty functions as diversity regularizers, our approach performs regularization over multiple locations of the posterior. The key driver of these determinantal penalties is a kernel function that induces repulsion in the latent model space domain. ER -
APA
Rockova, V., Moran, G. & George, E.. (2016). Determinantal Regularization for Ensemble Variable Selection. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 51:1105-1113 Available from https://proceedings.mlr.press/v51/rockova16.html.

Related Material