Free-Form Variational Inference for Gaussian Process State-Space Models

Xuhui Fan, Edwin V. Bonilla, Terence O’Kane, Scott A Sisson
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:9603-9622, 2023.

Abstract

Gaussian process state-space models (GPSSMs) provide a principled and flexible approach to modeling the dynamics of a latent state, which is observed at discrete-time points via a likelihood model. However, inference in GPSSMs is computationally and statistically challenging due to the large number of latent variables in the model and the strong temporal dependencies between them. In this paper, we propose a new method for inference in Bayesian GPSSMs, which overcomes the drawbacks of previous approaches, namely over-simplified assumptions, and high computational requirements. Our method is based on free-form variational inference via stochastic gradient Hamiltonian Monte Carlo within the inducing-variable formalism. Furthermore, by exploiting our proposed variational distribution, we provide a collapsed extension of our method where the inducing variables are marginalized analytically. We also showcase results when combining our framework with particle MCMC methods. We show that, on six real-world datasets, our approach can learn transition dynamics and latent states more accurately than competing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-fan23a, title = {Free-Form Variational Inference for {G}aussian Process State-Space Models}, author = {Fan, Xuhui and Bonilla, Edwin V. and O'Kane, Terence and Sisson, Scott A}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {9603--9622}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/fan23a/fan23a.pdf}, url = {https://proceedings.mlr.press/v202/fan23a.html}, abstract = {Gaussian process state-space models (GPSSMs) provide a principled and flexible approach to modeling the dynamics of a latent state, which is observed at discrete-time points via a likelihood model. However, inference in GPSSMs is computationally and statistically challenging due to the large number of latent variables in the model and the strong temporal dependencies between them. In this paper, we propose a new method for inference in Bayesian GPSSMs, which overcomes the drawbacks of previous approaches, namely over-simplified assumptions, and high computational requirements. Our method is based on free-form variational inference via stochastic gradient Hamiltonian Monte Carlo within the inducing-variable formalism. Furthermore, by exploiting our proposed variational distribution, we provide a collapsed extension of our method where the inducing variables are marginalized analytically. We also showcase results when combining our framework with particle MCMC methods. We show that, on six real-world datasets, our approach can learn transition dynamics and latent states more accurately than competing methods.} }
Endnote
%0 Conference Paper %T Free-Form Variational Inference for Gaussian Process State-Space Models %A Xuhui Fan %A Edwin V. Bonilla %A Terence O’Kane %A Scott A Sisson %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-fan23a %I PMLR %P 9603--9622 %U https://proceedings.mlr.press/v202/fan23a.html %V 202 %X Gaussian process state-space models (GPSSMs) provide a principled and flexible approach to modeling the dynamics of a latent state, which is observed at discrete-time points via a likelihood model. However, inference in GPSSMs is computationally and statistically challenging due to the large number of latent variables in the model and the strong temporal dependencies between them. In this paper, we propose a new method for inference in Bayesian GPSSMs, which overcomes the drawbacks of previous approaches, namely over-simplified assumptions, and high computational requirements. Our method is based on free-form variational inference via stochastic gradient Hamiltonian Monte Carlo within the inducing-variable formalism. Furthermore, by exploiting our proposed variational distribution, we provide a collapsed extension of our method where the inducing variables are marginalized analytically. We also showcase results when combining our framework with particle MCMC methods. We show that, on six real-world datasets, our approach can learn transition dynamics and latent states more accurately than competing methods.
APA
Fan, X., Bonilla, E.V., O’Kane, T. & Sisson, S.A.. (2023). Free-Form Variational Inference for Gaussian Process State-Space Models. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:9603-9622 Available from https://proceedings.mlr.press/v202/fan23a.html.

Related Material