Sparse-posterior Gaussian Processes for general likelihoods

Alan Qi, Ahmed Abdel-Gawad, Thomas Minka
Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, PMLR R8:457-464, 2010.

Abstract

Gaussian processes (GPs) provide a probabilistic nonparametric representation of functions in re- gression, classification, and other problems. Un- fortunately, exact learning with GPs is intractable for large datasets. A variety of approximate GP methods have been proposed that essentially map the large dataset into a small set of basis points. Among them, two state-of-the-art methods are sparse pseudo-input Gaussian process (SPGP) (Snelson and Ghahramani, 2006) and variable- sigma GP (VSGP) Walder et al. (2008), which generalizes SPGP and allows each basis point to have its own length scale. However, VSGP was only derived for regression. In this paper, we pro- pose a new sparse GP framework that uses expec- tation propagation to directly approximate gen- eral GP likelihoods using a sparse and smooth basis. It includes both SPGP and VSGP for re- gression as special cases. Plus as an EP algo- rithm, it inherits the ability to process data on- line. As a particular choice of approximating family, we blur each basis point with a Gaus- sian distribution that has a full covariance ma- trix representing the data distribution around that basis point; as a result, we can summarize local data manifold information with a small set of ba- sis points. Our experiments demonstrate that this framework outperforms previous GP classifica- tion methods on benchmark datasets in terms of minimizing divergence to the non-sparse GP so- lution as well as lower misclassification rate.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR8-qi10b, title = {Sparse-posterior {G}aussian Processes for general likelihoods}, author = {Qi, Alan and Abdel-Gawad, Ahmed and Minka, Thomas}, booktitle = {Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence}, pages = {457--464}, year = {2010}, editor = {Grünwald, Peter and Spirtes, Peter}, volume = {R8}, series = {Proceedings of Machine Learning Research}, month = {08--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r8/main/assets/qi10b/qi10b.pdf}, url = {https://proceedings.mlr.press/r8/qi10b.html}, abstract = {Gaussian processes (GPs) provide a probabilistic nonparametric representation of functions in re- gression, classification, and other problems. Un- fortunately, exact learning with GPs is intractable for large datasets. A variety of approximate GP methods have been proposed that essentially map the large dataset into a small set of basis points. Among them, two state-of-the-art methods are sparse pseudo-input Gaussian process (SPGP) (Snelson and Ghahramani, 2006) and variable- sigma GP (VSGP) Walder et al. (2008), which generalizes SPGP and allows each basis point to have its own length scale. However, VSGP was only derived for regression. In this paper, we pro- pose a new sparse GP framework that uses expec- tation propagation to directly approximate gen- eral GP likelihoods using a sparse and smooth basis. It includes both SPGP and VSGP for re- gression as special cases. Plus as an EP algo- rithm, it inherits the ability to process data on- line. As a particular choice of approximating family, we blur each basis point with a Gaus- sian distribution that has a full covariance ma- trix representing the data distribution around that basis point; as a result, we can summarize local data manifold information with a small set of ba- sis points. Our experiments demonstrate that this framework outperforms previous GP classifica- tion methods on benchmark datasets in terms of minimizing divergence to the non-sparse GP so- lution as well as lower misclassification rate.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Sparse-posterior Gaussian Processes for general likelihoods %A Alan Qi %A Ahmed Abdel-Gawad %A Thomas Minka %B Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2010 %E Peter Grünwald %E Peter Spirtes %F pmlr-vR8-qi10b %I PMLR %P 457--464 %U https://proceedings.mlr.press/r8/qi10b.html %V R8 %X Gaussian processes (GPs) provide a probabilistic nonparametric representation of functions in re- gression, classification, and other problems. Un- fortunately, exact learning with GPs is intractable for large datasets. A variety of approximate GP methods have been proposed that essentially map the large dataset into a small set of basis points. Among them, two state-of-the-art methods are sparse pseudo-input Gaussian process (SPGP) (Snelson and Ghahramani, 2006) and variable- sigma GP (VSGP) Walder et al. (2008), which generalizes SPGP and allows each basis point to have its own length scale. However, VSGP was only derived for regression. In this paper, we pro- pose a new sparse GP framework that uses expec- tation propagation to directly approximate gen- eral GP likelihoods using a sparse and smooth basis. It includes both SPGP and VSGP for re- gression as special cases. Plus as an EP algo- rithm, it inherits the ability to process data on- line. As a particular choice of approximating family, we blur each basis point with a Gaus- sian distribution that has a full covariance ma- trix representing the data distribution around that basis point; as a result, we can summarize local data manifold information with a small set of ba- sis points. Our experiments demonstrate that this framework outperforms previous GP classifica- tion methods on benchmark datasets in terms of minimizing divergence to the non-sparse GP so- lution as well as lower misclassification rate. %Z Reissued by PMLR on 04 October 2026.
APA
Qi, A., Abdel-Gawad, A. & Minka, T.. (2010). Sparse-posterior Gaussian Processes for general likelihoods. Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R8:457-464 Available from https://proceedings.mlr.press/r8/qi10b.html. Reissued by PMLR on 04 October 2026.

Related Material