Hierarchical Inducing Point Gaussian Process for Inter-domian Observations

Luhuan Wu, Andrew Miller, Lauren Anderson, Geoff Pleiss, David Blei, John Cunningham
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:2926-2934, 2021.

Abstract

We examine the general problem of inter-domain Gaussian Processes (GPs): problems where the GP realization and the noisy observations of that realization lie on different domains. When the mapping between those domains is linear, such as integration or differentiation, inference is still closed form. However, many of the scaling and approximation techniques that our community has developed do not apply to this setting. In this work, we introduce the hierarchical inducing point GP (HIP-GP), a scalable inter-domain GP inference method that enables us to improve the approximation accuracy by increasing the number of inducing points to the millions. HIP-GP, which relies on inducing points with grid structure and a stationary kernel assumption, is suitable for low-dimensional problems. In developing HIP-GP, we introduce (1) a fast whitening strategy, and (2) a novel preconditioner for conjugate gradients which can be helpful in general GP settings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-wu21b, title = { Hierarchical Inducing Point Gaussian Process for Inter-domian Observations }, author = {Wu, Luhuan and Miller, Andrew and Anderson, Lauren and Pleiss, Geoff and Blei, David and Cunningham, John}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {2926--2934}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/wu21b/wu21b.pdf}, url = {https://proceedings.mlr.press/v130/wu21b.html}, abstract = { We examine the general problem of inter-domain Gaussian Processes (GPs): problems where the GP realization and the noisy observations of that realization lie on different domains. When the mapping between those domains is linear, such as integration or differentiation, inference is still closed form. However, many of the scaling and approximation techniques that our community has developed do not apply to this setting. In this work, we introduce the hierarchical inducing point GP (HIP-GP), a scalable inter-domain GP inference method that enables us to improve the approximation accuracy by increasing the number of inducing points to the millions. HIP-GP, which relies on inducing points with grid structure and a stationary kernel assumption, is suitable for low-dimensional problems. In developing HIP-GP, we introduce (1) a fast whitening strategy, and (2) a novel preconditioner for conjugate gradients which can be helpful in general GP settings. } }
Endnote
%0 Conference Paper %T Hierarchical Inducing Point Gaussian Process for Inter-domian Observations %A Luhuan Wu %A Andrew Miller %A Lauren Anderson %A Geoff Pleiss %A David Blei %A John Cunningham %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-wu21b %I PMLR %P 2926--2934 %U https://proceedings.mlr.press/v130/wu21b.html %V 130 %X We examine the general problem of inter-domain Gaussian Processes (GPs): problems where the GP realization and the noisy observations of that realization lie on different domains. When the mapping between those domains is linear, such as integration or differentiation, inference is still closed form. However, many of the scaling and approximation techniques that our community has developed do not apply to this setting. In this work, we introduce the hierarchical inducing point GP (HIP-GP), a scalable inter-domain GP inference method that enables us to improve the approximation accuracy by increasing the number of inducing points to the millions. HIP-GP, which relies on inducing points with grid structure and a stationary kernel assumption, is suitable for low-dimensional problems. In developing HIP-GP, we introduce (1) a fast whitening strategy, and (2) a novel preconditioner for conjugate gradients which can be helpful in general GP settings.
APA
Wu, L., Miller, A., Anderson, L., Pleiss, G., Blei, D. & Cunningham, J.. (2021). Hierarchical Inducing Point Gaussian Process for Inter-domian Observations . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:2926-2934 Available from https://proceedings.mlr.press/v130/wu21b.html.

Related Material