Scalable Exact Inference in Multi-Output Gaussian Processes

Wessel Bruinsma, Eric Perim, William Tebbutt, Scott Hosking, Arno Solin, Richard Turner
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:1190-1201, 2020.

Abstract

Multi-output Gaussian processes (MOGPs) leverage the flexibility and interpretability of GPs while capturing structure across outputs, which is desirable, for example, in spatio-temporal modelling. The key problem with MOGPs is their computational scaling O(n3p3), which is cubic in the number of both inputs n (e.g., time points or locations) and outputs p. For this reason, a popular class of MOGPs assumes that the data live around a low-dimensional linear subspace, reducing the complexity to O(n3m3). However, this cost is still cubic in the dimensionality of the subspace m, which is still prohibitively expensive for many applications. We propose the use of a sufficient statistic of the data to accelerate inference and learning in MOGPs with orthogonal bases. The method achieves linear scaling in m in practice, allowing these models to scale to large m without sacrificing significant expressivity or requiring approximation. This advance opens up a wide range of real-world tasks and can be combined with existing GP approximations in a plug-and-play way. We demonstrate the efficacy of the method on various synthetic and real-world data sets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-bruinsma20a, title = {Scalable Exact Inference in Multi-Output {G}aussian Processes}, author = {Bruinsma, Wessel and Perim, Eric and Tebbutt, William and Hosking, Scott and Solin, Arno and Turner, Richard}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {1190--1201}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/bruinsma20a/bruinsma20a.pdf}, url = {https://proceedings.mlr.press/v119/bruinsma20a.html}, abstract = {Multi-output Gaussian processes (MOGPs) leverage the flexibility and interpretability of GPs while capturing structure across outputs, which is desirable, for example, in spatio-temporal modelling. The key problem with MOGPs is their computational scaling $O(n^3 p^3)$, which is cubic in the number of both inputs $n$ (e.g., time points or locations) and outputs $p$. For this reason, a popular class of MOGPs assumes that the data live around a low-dimensional linear subspace, reducing the complexity to $O(n^3 m^3)$. However, this cost is still cubic in the dimensionality of the subspace $m$, which is still prohibitively expensive for many applications. We propose the use of a sufficient statistic of the data to accelerate inference and learning in MOGPs with orthogonal bases. The method achieves linear scaling in $m$ in practice, allowing these models to scale to large $m$ without sacrificing significant expressivity or requiring approximation. This advance opens up a wide range of real-world tasks and can be combined with existing GP approximations in a plug-and-play way. We demonstrate the efficacy of the method on various synthetic and real-world data sets.} }
Endnote
%0 Conference Paper %T Scalable Exact Inference in Multi-Output Gaussian Processes %A Wessel Bruinsma %A Eric Perim %A William Tebbutt %A Scott Hosking %A Arno Solin %A Richard Turner %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-bruinsma20a %I PMLR %P 1190--1201 %U https://proceedings.mlr.press/v119/bruinsma20a.html %V 119 %X Multi-output Gaussian processes (MOGPs) leverage the flexibility and interpretability of GPs while capturing structure across outputs, which is desirable, for example, in spatio-temporal modelling. The key problem with MOGPs is their computational scaling $O(n^3 p^3)$, which is cubic in the number of both inputs $n$ (e.g., time points or locations) and outputs $p$. For this reason, a popular class of MOGPs assumes that the data live around a low-dimensional linear subspace, reducing the complexity to $O(n^3 m^3)$. However, this cost is still cubic in the dimensionality of the subspace $m$, which is still prohibitively expensive for many applications. We propose the use of a sufficient statistic of the data to accelerate inference and learning in MOGPs with orthogonal bases. The method achieves linear scaling in $m$ in practice, allowing these models to scale to large $m$ without sacrificing significant expressivity or requiring approximation. This advance opens up a wide range of real-world tasks and can be combined with existing GP approximations in a plug-and-play way. We demonstrate the efficacy of the method on various synthetic and real-world data sets.
APA
Bruinsma, W., Perim, E., Tebbutt, W., Hosking, S., Solin, A. & Turner, R.. (2020). Scalable Exact Inference in Multi-Output Gaussian Processes. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:1190-1201 Available from https://proceedings.mlr.press/v119/bruinsma20a.html.

Related Material