Additive Gaussian Processes Revisited

Xiaoyu Lu, Alexis Boukouvalas, James Hensman
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:14358-14383, 2022.

Abstract

Gaussian Process (GP) models are a class of flexible non-parametric models that have rich representational power. By using a Gaussian process with additive structure, complex responses can be modelled whilst retaining interpretability. Previous work showed that additive Gaussian process models require high-dimensional interaction terms. We propose the orthogonal additive kernel (OAK), which imposes an orthogonality constraint on the additive functions, enabling an identifiable, low-dimensional representation of the functional relationship. We connect the OAK kernel to functional ANOVA decomposition, and show improved convergence rates for sparse computation methods. With only a small number of additive low-dimensional terms, we demonstrate the OAK model achieves similar or better predictive performance compared to black-box models, while retaining interpretability.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-lu22b, title = {Additive {G}aussian Processes Revisited}, author = {Lu, Xiaoyu and Boukouvalas, Alexis and Hensman, James}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {14358--14383}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/lu22b/lu22b.pdf}, url = {https://proceedings.mlr.press/v162/lu22b.html}, abstract = {Gaussian Process (GP) models are a class of flexible non-parametric models that have rich representational power. By using a Gaussian process with additive structure, complex responses can be modelled whilst retaining interpretability. Previous work showed that additive Gaussian process models require high-dimensional interaction terms. We propose the orthogonal additive kernel (OAK), which imposes an orthogonality constraint on the additive functions, enabling an identifiable, low-dimensional representation of the functional relationship. We connect the OAK kernel to functional ANOVA decomposition, and show improved convergence rates for sparse computation methods. With only a small number of additive low-dimensional terms, we demonstrate the OAK model achieves similar or better predictive performance compared to black-box models, while retaining interpretability.} }
Endnote
%0 Conference Paper %T Additive Gaussian Processes Revisited %A Xiaoyu Lu %A Alexis Boukouvalas %A James Hensman %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-lu22b %I PMLR %P 14358--14383 %U https://proceedings.mlr.press/v162/lu22b.html %V 162 %X Gaussian Process (GP) models are a class of flexible non-parametric models that have rich representational power. By using a Gaussian process with additive structure, complex responses can be modelled whilst retaining interpretability. Previous work showed that additive Gaussian process models require high-dimensional interaction terms. We propose the orthogonal additive kernel (OAK), which imposes an orthogonality constraint on the additive functions, enabling an identifiable, low-dimensional representation of the functional relationship. We connect the OAK kernel to functional ANOVA decomposition, and show improved convergence rates for sparse computation methods. With only a small number of additive low-dimensional terms, we demonstrate the OAK model achieves similar or better predictive performance compared to black-box models, while retaining interpretability.
APA
Lu, X., Boukouvalas, A. & Hensman, J.. (2022). Additive Gaussian Processes Revisited. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:14358-14383 Available from https://proceedings.mlr.press/v162/lu22b.html.

Related Material