AutoGP: Exploring the Capabilities and Limitations of Gaussian Process Models

Karl Krauth, Edwin V. Bonilla, Kurt Cutajar, Maurizio Filippone
Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, PMLR R15:121-130, 2017.

Abstract

We investigate the capabilities and limitations of Gaussian process (GP) models by jointly exploring three complementary directions: (i) scalable and statistically efficient inference; (ii) flexible kernels; and (iii) objective functions for hyperparameter learning alternative to the marginal likelihood. Our approach outperforms all previous GP methods on the MNIST dataset; performs comparatively to kernel-based methods using the RECTANGLES-IMAGE dataset; and breaks the 1% error-rate barrier in GP models on the MNIST8M dataset, while showing unprecedented scalability (8 million observations) in GP classification. Overall, our approach represents a significant breakthrough in kernel methods and GP models, bridging the gap between deep learning and kernel machines.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR15-krauth17a, title = {AutoGP: Exploring the Capabilities and Limitations of {G}aussian Process Models}, author = {Krauth, Karl and Bonilla, Edwin V. and Cutajar, Kurt and Filippone, Maurizio}, booktitle = {Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence}, pages = {121--130}, year = {2017}, editor = {Elidan, Gal and Kersting, Kristian}, volume = {R15}, series = {Proceedings of Machine Learning Research}, month = {11--15 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r15/main/assets/krauth17a/krauth17a.pdf}, url = {https://proceedings.mlr.press/r15/krauth17a.html}, abstract = {We investigate the capabilities and limitations of Gaussian process (GP) models by jointly exploring three complementary directions: (i) scalable and statistically efficient inference; (ii) flexible kernels; and (iii) objective functions for hyperparameter learning alternative to the marginal likelihood. Our approach outperforms all previous GP methods on the MNIST dataset; performs comparatively to kernel-based methods using the RECTANGLES-IMAGE dataset; and breaks the 1% error-rate barrier in GP models on the MNIST8M dataset, while showing unprecedented scalability (8 million observations) in GP classification. Overall, our approach represents a significant breakthrough in kernel methods and GP models, bridging the gap between deep learning and kernel machines.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T AutoGP: Exploring the Capabilities and Limitations of Gaussian Process Models %A Karl Krauth %A Edwin V. Bonilla %A Kurt Cutajar %A Maurizio Filippone %B Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2017 %E Gal Elidan %E Kristian Kersting %F pmlr-vR15-krauth17a %I PMLR %P 121--130 %U https://proceedings.mlr.press/r15/krauth17a.html %V R15 %X We investigate the capabilities and limitations of Gaussian process (GP) models by jointly exploring three complementary directions: (i) scalable and statistically efficient inference; (ii) flexible kernels; and (iii) objective functions for hyperparameter learning alternative to the marginal likelihood. Our approach outperforms all previous GP methods on the MNIST dataset; performs comparatively to kernel-based methods using the RECTANGLES-IMAGE dataset; and breaks the 1% error-rate barrier in GP models on the MNIST8M dataset, while showing unprecedented scalability (8 million observations) in GP classification. Overall, our approach represents a significant breakthrough in kernel methods and GP models, bridging the gap between deep learning and kernel machines. %Z Reissued by PMLR on 04 October 2026.
APA
Krauth, K., Bonilla, E.V., Cutajar, K. & Filippone, M.. (2017). AutoGP: Exploring the Capabilities and Limitations of Gaussian Process Models. Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R15:121-130 Available from https://proceedings.mlr.press/r15/krauth17a.html. Reissued by PMLR on 04 October 2026.

Related Material