Measuring the robustness of Gaussian processes to kernel choice

William T. Stephenson, Soumya Ghosh, Tin D. Nguyen, Mikhail Yurochkin, Sameer Deshpande, Tamara Broderick
Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, PMLR 151:3308-3331, 2022.

Abstract

Gaussian processes (GPs) are used to make medical and scientific decisions, including in cardiac care and monitoring of carbon dioxide emissions. Notably, the choice of GP kernel is often somewhat arbitrary. In particular, uncountably many kernels typically align with qualitative prior knowledge (e.g. function smoothness or stationarity). But in practice, data analysts choose among a handful of convenient standard kernels (e.g. squared exponential). In the present work, we ask: Would decisions made with a GP differ under other, qualitatively interchangeable kernels? We show how to formulate this sensitivity analysis as a constrained optimization problem over a finite-dimensional space. We can then use standard optimizers to identify substantive changes in relevant decisions made with a GP. We demonstrate in both synthetic and real-world examples that decisions made with a GP can exhibit substantial sensitivity to kernel choice, even when prior draws are qualitatively interchangeable to a user.

Cite this Paper

BibTeX
@InProceedings{pmlr-v151-stephenson22a, title = { Measuring the robustness of Gaussian processes to kernel choice }, author = {Stephenson, William T. and Ghosh, Soumya and Nguyen, Tin D. and Yurochkin, Mikhail and Deshpande, Sameer and Broderick, Tamara}, booktitle = {Proceedings of The 25th International Conference on Artificial Intelligence and Statistics}, pages = {3308--3331}, year = {2022}, editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel}, volume = {151}, series = {Proceedings of Machine Learning Research}, month = {28--30 Mar}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v151/stephenson22a/stephenson22a.pdf}, url = {https://proceedings.mlr.press/v151/stephenson22a.html}, abstract = { Gaussian processes (GPs) are used to make medical and scientific decisions, including in cardiac care and monitoring of carbon dioxide emissions. Notably, the choice of GP kernel is often somewhat arbitrary. In particular, uncountably many kernels typically align with qualitative prior knowledge (e.g. function smoothness or stationarity). But in practice, data analysts choose among a handful of convenient standard kernels (e.g. squared exponential). In the present work, we ask: Would decisions made with a GP differ under other, qualitatively interchangeable kernels? We show how to formulate this sensitivity analysis as a constrained optimization problem over a finite-dimensional space. We can then use standard optimizers to identify substantive changes in relevant decisions made with a GP. We demonstrate in both synthetic and real-world examples that decisions made with a GP can exhibit substantial sensitivity to kernel choice, even when prior draws are qualitatively interchangeable to a user. } }
Endnote
%0 Conference Paper %T Measuring the robustness of Gaussian processes to kernel choice %A William T. Stephenson %A Soumya Ghosh %A Tin D. Nguyen %A Mikhail Yurochkin %A Sameer Deshpande %A Tamara Broderick %B Proceedings of The 25th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2022 %E Gustau Camps-Valls %E Francisco J. R. Ruiz %E Isabel Valera %F pmlr-v151-stephenson22a %I PMLR %P 3308--3331 %U https://proceedings.mlr.press/v151/stephenson22a.html %V 151 %X Gaussian processes (GPs) are used to make medical and scientific decisions, including in cardiac care and monitoring of carbon dioxide emissions. Notably, the choice of GP kernel is often somewhat arbitrary. In particular, uncountably many kernels typically align with qualitative prior knowledge (e.g. function smoothness or stationarity). But in practice, data analysts choose among a handful of convenient standard kernels (e.g. squared exponential). In the present work, we ask: Would decisions made with a GP differ under other, qualitatively interchangeable kernels? We show how to formulate this sensitivity analysis as a constrained optimization problem over a finite-dimensional space. We can then use standard optimizers to identify substantive changes in relevant decisions made with a GP. We demonstrate in both synthetic and real-world examples that decisions made with a GP can exhibit substantial sensitivity to kernel choice, even when prior draws are qualitatively interchangeable to a user.
APA
Stephenson, W.T., Ghosh, S., Nguyen, T.D., Yurochkin, M., Deshpande, S. & Broderick, T.. (2022). Measuring the robustness of Gaussian processes to kernel choice . Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 151:3308-3331 Available from https://proceedings.mlr.press/v151/stephenson22a.html.

Related Material