State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes

William Wilkinson, Paul Chang, Michael Andersen, Arno Solin
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:10270-10281, 2020.

Abstract

We formulate approximate Bayesian inference in non-conjugate temporal and spatio-temporal Gaussian process models as a simple parameter update rule applied during Kalman smoothing. This viewpoint encompasses most inference schemes, including expectation propagation (EP), the classical (Extended, Unscented, etc.) Kalman smoothers, and variational inference. We provide a unifying perspective on these algorithms, showing how replacing the power EP moment matching step with linearisation recovers the classical smoothers. EP provides some benefits over the traditional methods via introduction of the so-called cavity distribution, and we combine these benefits with the computational efficiency of linearisation, providing extensive empirical analysis demonstrating the efficacy of various algorithms under this unifying framework. We provide a fast implementation of all methods in JAX.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-wilkinson20a, title = {State Space Expectation Propagation: Efficient Inference Schemes for Temporal {G}aussian Processes}, author = {Wilkinson, William and Chang, Paul and Andersen, Michael and Solin, Arno}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {10270--10281}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/wilkinson20a/wilkinson20a.pdf}, url = {https://proceedings.mlr.press/v119/wilkinson20a.html}, abstract = {We formulate approximate Bayesian inference in non-conjugate temporal and spatio-temporal Gaussian process models as a simple parameter update rule applied during Kalman smoothing. This viewpoint encompasses most inference schemes, including expectation propagation (EP), the classical (Extended, Unscented, etc.) Kalman smoothers, and variational inference. We provide a unifying perspective on these algorithms, showing how replacing the power EP moment matching step with linearisation recovers the classical smoothers. EP provides some benefits over the traditional methods via introduction of the so-called cavity distribution, and we combine these benefits with the computational efficiency of linearisation, providing extensive empirical analysis demonstrating the efficacy of various algorithms under this unifying framework. We provide a fast implementation of all methods in JAX.} }
Endnote
%0 Conference Paper %T State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes %A William Wilkinson %A Paul Chang %A Michael Andersen %A Arno Solin %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-wilkinson20a %I PMLR %P 10270--10281 %U https://proceedings.mlr.press/v119/wilkinson20a.html %V 119 %X We formulate approximate Bayesian inference in non-conjugate temporal and spatio-temporal Gaussian process models as a simple parameter update rule applied during Kalman smoothing. This viewpoint encompasses most inference schemes, including expectation propagation (EP), the classical (Extended, Unscented, etc.) Kalman smoothers, and variational inference. We provide a unifying perspective on these algorithms, showing how replacing the power EP moment matching step with linearisation recovers the classical smoothers. EP provides some benefits over the traditional methods via introduction of the so-called cavity distribution, and we combine these benefits with the computational efficiency of linearisation, providing extensive empirical analysis demonstrating the efficacy of various algorithms under this unifying framework. We provide a fast implementation of all methods in JAX.
APA
Wilkinson, W., Chang, P., Andersen, M. & Solin, A.. (2020). State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:10270-10281 Available from https://proceedings.mlr.press/v119/wilkinson20a.html.

Related Material