Robust and Conjugate Gaussian Process Regression

Matias Altamirano, Francois-Xavier Briol, Jeremias Knoblauch
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:1155-1185, 2024.

Abstract

To enable closed form conditioning, a common assumption in Gaussian process (GP) regression is independent and identically distributed Gaussian observation noise. This strong and simplistic assumption is often violated in practice, which leads to unreliable inferences and uncertainty quantification. Unfortunately, existing methods for robustifying GPs break closed-form conditioning, which makes them less attractive to practitioners and significantly more computationally expensive. In this paper, we demonstrate how to perform provably robust and conjugate Gaussian process (RCGP) regression at virtually no additional cost using generalised Bayesian inference. RCGP is particularly versatile as it enables exact conjugate closed form updates in all settings where standard GPs admit them. To demonstrate its strong empirical performance, we deploy RCGP for problems ranging from Bayesian optimisation to sparse variational Gaussian processes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-altamirano24a, title = {Robust and Conjugate {G}aussian Process Regression}, author = {Altamirano, Matias and Briol, Francois-Xavier and Knoblauch, Jeremias}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {1155--1185}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/altamirano24a/altamirano24a.pdf}, url = {https://proceedings.mlr.press/v235/altamirano24a.html}, abstract = {To enable closed form conditioning, a common assumption in Gaussian process (GP) regression is independent and identically distributed Gaussian observation noise. This strong and simplistic assumption is often violated in practice, which leads to unreliable inferences and uncertainty quantification. Unfortunately, existing methods for robustifying GPs break closed-form conditioning, which makes them less attractive to practitioners and significantly more computationally expensive. In this paper, we demonstrate how to perform provably robust and conjugate Gaussian process (RCGP) regression at virtually no additional cost using generalised Bayesian inference. RCGP is particularly versatile as it enables exact conjugate closed form updates in all settings where standard GPs admit them. To demonstrate its strong empirical performance, we deploy RCGP for problems ranging from Bayesian optimisation to sparse variational Gaussian processes.} }
Endnote
%0 Conference Paper %T Robust and Conjugate Gaussian Process Regression %A Matias Altamirano %A Francois-Xavier Briol %A Jeremias Knoblauch %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-altamirano24a %I PMLR %P 1155--1185 %U https://proceedings.mlr.press/v235/altamirano24a.html %V 235 %X To enable closed form conditioning, a common assumption in Gaussian process (GP) regression is independent and identically distributed Gaussian observation noise. This strong and simplistic assumption is often violated in practice, which leads to unreliable inferences and uncertainty quantification. Unfortunately, existing methods for robustifying GPs break closed-form conditioning, which makes them less attractive to practitioners and significantly more computationally expensive. In this paper, we demonstrate how to perform provably robust and conjugate Gaussian process (RCGP) regression at virtually no additional cost using generalised Bayesian inference. RCGP is particularly versatile as it enables exact conjugate closed form updates in all settings where standard GPs admit them. To demonstrate its strong empirical performance, we deploy RCGP for problems ranging from Bayesian optimisation to sparse variational Gaussian processes.
APA
Altamirano, M., Briol, F. & Knoblauch, J.. (2024). Robust and Conjugate Gaussian Process Regression. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:1155-1185 Available from https://proceedings.mlr.press/v235/altamirano24a.html.

Related Material