Approximate Inference for Fully Bayesian Gaussian Process Regression
[edit]
Proceedings of The 2nd Symposium on Advances in Approximate Bayesian Inference, PMLR 118:112, 2020.
Abstract
Learning in Gaussian Process models occurs through the adaptation of hyperparameters of the mean and the covariance function. The classical approach entails maximizing the marginal likelihood yielding fixed point estimates (an approach called Type II maximum likelihood or MLII). An alternative learning procedure is to infer the posterior over hyperparameters in a hierarchical specication of GPs we call Fully Bayesian Gaussian Process Regression (GPR). This work considers two approximation schemes for the intractable hyperparameter posterior: 1) Hamiltonian Monte Carlo (HMC) yielding a sampling based approximation and 2) Variational Inference (VI) where the posterior over hyperparameters is approximated by a factorized Gaussian (meanfield) or a fullrank Gaussian accounting for correlations between hyperparameters. We analyse the predictive performance for fully Bayesian GPR on a range of benchmark data sets.
Related Material


