High-Dimensional Gaussian Process Inference with Derivatives

Filip De Roos, Alexandra Gessner, Philipp Hennig
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:2535-2545, 2021.

Abstract

Although it is widely known that Gaussian processes can be conditioned on observations of the gradient, this functionality is of limited use due to the prohibitive computational cost of $\mathcal{O}(N^3 D^3)$ in data points $N$ and dimension $D$. The dilemma of gradient observations is that a single one of them comes at the same cost as $D$ independent function evaluations, so the latter are often preferred. Careful scrutiny reveals, however, that derivative observations give rise to highly structured kernel Gram matrices for very general classes of kernels (inter alia, stationary kernels). We show that in the \emph{low-data} regime $N

Cite this Paper


BibTeX
@InProceedings{pmlr-v139-de-roos21a, title = {High-Dimensional Gaussian Process Inference with Derivatives}, author = {De Roos, Filip and Gessner, Alexandra and Hennig, Philipp}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {2535--2545}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/de-roos21a/de-roos21a.pdf}, url = {https://proceedings.mlr.press/v139/de-roos21a.html}, abstract = {Although it is widely known that Gaussian processes can be conditioned on observations of the gradient, this functionality is of limited use due to the prohibitive computational cost of $\mathcal{O}(N^3 D^3)$ in data points $N$ and dimension $D$. The dilemma of gradient observations is that a single one of them comes at the same cost as $D$ independent function evaluations, so the latter are often preferred. Careful scrutiny reveals, however, that derivative observations give rise to highly structured kernel Gram matrices for very general classes of kernels (inter alia, stationary kernels). We show that in the \emph{low-data} regime $N
Endnote
%0 Conference Paper %T High-Dimensional Gaussian Process Inference with Derivatives %A Filip De Roos %A Alexandra Gessner %A Philipp Hennig %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-de-roos21a %I PMLR %P 2535--2545 %U https://proceedings.mlr.press/v139/de-roos21a.html %V 139 %X Although it is widely known that Gaussian processes can be conditioned on observations of the gradient, this functionality is of limited use due to the prohibitive computational cost of $\mathcal{O}(N^3 D^3)$ in data points $N$ and dimension $D$. The dilemma of gradient observations is that a single one of them comes at the same cost as $D$ independent function evaluations, so the latter are often preferred. Careful scrutiny reveals, however, that derivative observations give rise to highly structured kernel Gram matrices for very general classes of kernels (inter alia, stationary kernels). We show that in the \emph{low-data} regime $N
APA
De Roos, F., Gessner, A. & Hennig, P.. (2021). High-Dimensional Gaussian Process Inference with Derivatives. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:2535-2545 Available from https://proceedings.mlr.press/v139/de-roos21a.html.

Related Material