Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations

Gaussian processes (GP) are Bayesian non- parametric models that are widely used for prob- abilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time cost in the data size. This paper presents two parallel GP re- gression methods that exploit low-rank covari- ance matrix approximations for distributing the computational load among parallel machines to achieve time efficiency and scalability. We the- oretically guarantee the predictive performances of our proposed parallel GPs to be equivalent to that of some centralized approximate GP regres- sion methods: The computation of their central- ized counterparts can be distributed among par- allel machines, hence achieving greater time effi- ciency and scalability. We analytically compare the properties of our parallel GPs such as time, space, and communication complexity. Empir- ical evaluation on two real-world datasets in a cluster of 20 computing nodes shows that our parallel GPs are significantly more time-efficient and scalable than their centralized counterparts and exact/full GP while achieving predictive per- formances comparable to full GP.