Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution

Topi Paananen, Juho Piironen, Michael Riis Andersen, Aki Vehtari
Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, PMLR 89:1743-1752, 2019.

Abstract

Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse length-scale parameter of each input variable as a proxy for variable relevance. This implicitly determined relevance has several drawbacks that prevent the selection of optimal input variables in terms of predictive performance. To improve on this, we propose two novel variable selection methods for Gaussian process models that utilize the predictions of a full model in the vicinity of the training points and thereby rank the variables based on their predictive relevance. Our empirical results on synthetic and real world data sets demonstrate improved variable selection compared to automatic relevance determination in terms of variability and predictive performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v89-paananen19a, title = {Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution}, author = {Paananen, Topi and Piironen, Juho and Andersen, Michael Riis and Vehtari, Aki}, booktitle = {Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics}, pages = {1743--1752}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/paananen19a/paananen19a.pdf}, url = {https://proceedings.mlr.press/v89/paananen19a.html}, abstract = {Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse length-scale parameter of each input variable as a proxy for variable relevance. This implicitly determined relevance has several drawbacks that prevent the selection of optimal input variables in terms of predictive performance. To improve on this, we propose two novel variable selection methods for Gaussian process models that utilize the predictions of a full model in the vicinity of the training points and thereby rank the variables based on their predictive relevance. Our empirical results on synthetic and real world data sets demonstrate improved variable selection compared to automatic relevance determination in terms of variability and predictive performance.} }
Endnote
%0 Conference Paper %T Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution %A Topi Paananen %A Juho Piironen %A Michael Riis Andersen %A Aki Vehtari %B Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Masashi Sugiyama %F pmlr-v89-paananen19a %I PMLR %P 1743--1752 %U https://proceedings.mlr.press/v89/paananen19a.html %V 89 %X Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse length-scale parameter of each input variable as a proxy for variable relevance. This implicitly determined relevance has several drawbacks that prevent the selection of optimal input variables in terms of predictive performance. To improve on this, we propose two novel variable selection methods for Gaussian process models that utilize the predictions of a full model in the vicinity of the training points and thereby rank the variables based on their predictive relevance. Our empirical results on synthetic and real world data sets demonstrate improved variable selection compared to automatic relevance determination in terms of variability and predictive performance.
APA
Paananen, T., Piironen, J., Andersen, M.R. & Vehtari, A.. (2019). Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution. Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 89:1743-1752 Available from https://proceedings.mlr.press/v89/paananen19a.html.

Related Material