Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models

Theo Galy-Fajou, Florian Wenzel, Manfred Opper
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR 108:3025-3035, 2020.

Abstract

We propose automated augmented conjugate inference, a new inference method for non-conjugate Gaussian processes (GP) models.Our method automatically constructs an auxiliary variable augmentation that renders the GP model conditionally conjugate. Building on the conjugate structure of the augmented model, we develop two inference methods. First, a fast and scalable stochastic variational inference method that uses efficient block coordinate ascent updates, which are computed in closed form. Second, an asymptotically correct Gibbs sampler that is useful for small datasets.Our experiments show that our method is up two orders of magnitude faster and more robust than existing state-of-the-art black-box methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v108-galy-fajou20a, title = {Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models}, author = {Galy-Fajou, Theo and Wenzel, Florian and Opper, Manfred}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {3025--3035}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/galy-fajou20a/galy-fajou20a.pdf}, url = {https://proceedings.mlr.press/v108/galy-fajou20a.html}, abstract = {We propose automated augmented conjugate inference, a new inference method for non-conjugate Gaussian processes (GP) models.Our method automatically constructs an auxiliary variable augmentation that renders the GP model conditionally conjugate. Building on the conjugate structure of the augmented model, we develop two inference methods. First, a fast and scalable stochastic variational inference method that uses efficient block coordinate ascent updates, which are computed in closed form. Second, an asymptotically correct Gibbs sampler that is useful for small datasets.Our experiments show that our method is up two orders of magnitude faster and more robust than existing state-of-the-art black-box methods.} }
Endnote
%0 Conference Paper %T Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models %A Theo Galy-Fajou %A Florian Wenzel %A Manfred Opper %B Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2020 %E Silvia Chiappa %E Roberto Calandra %F pmlr-v108-galy-fajou20a %I PMLR %P 3025--3035 %U https://proceedings.mlr.press/v108/galy-fajou20a.html %V 108 %X We propose automated augmented conjugate inference, a new inference method for non-conjugate Gaussian processes (GP) models.Our method automatically constructs an auxiliary variable augmentation that renders the GP model conditionally conjugate. Building on the conjugate structure of the augmented model, we develop two inference methods. First, a fast and scalable stochastic variational inference method that uses efficient block coordinate ascent updates, which are computed in closed form. Second, an asymptotically correct Gibbs sampler that is useful for small datasets.Our experiments show that our method is up two orders of magnitude faster and more robust than existing state-of-the-art black-box methods.
APA
Galy-Fajou, T., Wenzel, F. & Opper, M.. (2020). Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 108:3025-3035 Available from https://proceedings.mlr.press/v108/galy-fajou20a.html.

Related Material