Probabilistic selection of inducing points in sparse Gaussian processes

Anders Kirk Uhrenholt, Valentin Charvet, Bjørn Sand Jensen
Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, PMLR 161:1035-1044, 2021.

Abstract

Sparse Gaussian processes and various extensions thereof are enabled through inducing points, that simultaneously bottleneck the predictive capacity and act as the main contributor towards model complexity. However, the number of inducing points is generally not associated with uncertainty which prevents us from applying the apparatus of Bayesian reasoning for identifying an appropriate trade-off. In this work we place a point process prior on the inducing points and approximate the associated posterior through stochastic variational inference. By letting the prior encourage a moderate number of inducing points, we enable the model to learn which and how many points to utilise. We experimentally show that fewer inducing points are preferred by the model as the points become less informative, and further demonstrate how the method can be employed in deep Gaussian processes and latent variable modelling.

Cite this Paper

BibTeX
@InProceedings{pmlr-v161-uhrenholt21a, title = {Probabilistic selection of inducing points in sparse Gaussian processes}, author = {Uhrenholt, Anders Kirk and Charvet, Valentin and Jensen, Bj{\o}rn Sand}, booktitle = {Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence}, pages = {1035--1044}, year = {2021}, editor = {de Campos, Cassio and Maathuis, Marloes H.}, volume = {161}, series = {Proceedings of Machine Learning Research}, month = {27--30 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v161/uhrenholt21a/uhrenholt21a.pdf}, url = {https://proceedings.mlr.press/v161/uhrenholt21a.html}, abstract = {Sparse Gaussian processes and various extensions thereof are enabled through inducing points, that simultaneously bottleneck the predictive capacity and act as the main contributor towards model complexity. However, the number of inducing points is generally not associated with uncertainty which prevents us from applying the apparatus of Bayesian reasoning for identifying an appropriate trade-off. In this work we place a point process prior on the inducing points and approximate the associated posterior through stochastic variational inference. By letting the prior encourage a moderate number of inducing points, we enable the model to learn which and how many points to utilise. We experimentally show that fewer inducing points are preferred by the model as the points become less informative, and further demonstrate how the method can be employed in deep Gaussian processes and latent variable modelling.} }
Endnote
%0 Conference Paper %T Probabilistic selection of inducing points in sparse Gaussian processes %A Anders Kirk Uhrenholt %A Valentin Charvet %A Bjørn Sand Jensen %B Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2021 %E Cassio de Campos %E Marloes H. Maathuis %F pmlr-v161-uhrenholt21a %I PMLR %P 1035--1044 %U https://proceedings.mlr.press/v161/uhrenholt21a.html %V 161 %X Sparse Gaussian processes and various extensions thereof are enabled through inducing points, that simultaneously bottleneck the predictive capacity and act as the main contributor towards model complexity. However, the number of inducing points is generally not associated with uncertainty which prevents us from applying the apparatus of Bayesian reasoning for identifying an appropriate trade-off. In this work we place a point process prior on the inducing points and approximate the associated posterior through stochastic variational inference. By letting the prior encourage a moderate number of inducing points, we enable the model to learn which and how many points to utilise. We experimentally show that fewer inducing points are preferred by the model as the points become less informative, and further demonstrate how the method can be employed in deep Gaussian processes and latent variable modelling.
APA
Uhrenholt, A.K., Charvet, V. & Jensen, B.S.. (2021). Probabilistic selection of inducing points in sparse Gaussian processes. Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 161:1035-1044 Available from https://proceedings.mlr.press/v161/uhrenholt21a.html.

Related Material