Variational Gaussian Process Diffusion Processes

Prakhar Verma, Vincent Adam, Arno Solin
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:1909-1917, 2024.

Abstract

Diffusion processes are a class of stochastic differential equations (SDEs) providing a rich family of expressive models that arise naturally in dynamic modelling tasks. Probabilistic inference and learning under generative models with latent processes endowed with a non-linear diffusion process prior are intractable problems. We build upon work within variational inference, approximating the posterior process as a linear diffusion process, and point out pathologies in the approach. We propose an alternative parameterization of the Gaussian variational process using a site-based exponential family description. This allows us to trade a slow inference algorithm with fixed-point iterations for a fast algorithm for convex optimization akin to natural gradient descent, which also provides a better objective for learning model parameters.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-verma24a, title = {Variational {G}aussian Process Diffusion Processes}, author = {Verma, Prakhar and Adam, Vincent and Solin, Arno}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {1909--1917}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/verma24a/verma24a.pdf}, url = {https://proceedings.mlr.press/v238/verma24a.html}, abstract = {Diffusion processes are a class of stochastic differential equations (SDEs) providing a rich family of expressive models that arise naturally in dynamic modelling tasks. Probabilistic inference and learning under generative models with latent processes endowed with a non-linear diffusion process prior are intractable problems. We build upon work within variational inference, approximating the posterior process as a linear diffusion process, and point out pathologies in the approach. We propose an alternative parameterization of the Gaussian variational process using a site-based exponential family description. This allows us to trade a slow inference algorithm with fixed-point iterations for a fast algorithm for convex optimization akin to natural gradient descent, which also provides a better objective for learning model parameters.} }
Endnote
%0 Conference Paper %T Variational Gaussian Process Diffusion Processes %A Prakhar Verma %A Vincent Adam %A Arno Solin %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-verma24a %I PMLR %P 1909--1917 %U https://proceedings.mlr.press/v238/verma24a.html %V 238 %X Diffusion processes are a class of stochastic differential equations (SDEs) providing a rich family of expressive models that arise naturally in dynamic modelling tasks. Probabilistic inference and learning under generative models with latent processes endowed with a non-linear diffusion process prior are intractable problems. We build upon work within variational inference, approximating the posterior process as a linear diffusion process, and point out pathologies in the approach. We propose an alternative parameterization of the Gaussian variational process using a site-based exponential family description. This allows us to trade a slow inference algorithm with fixed-point iterations for a fast algorithm for convex optimization akin to natural gradient descent, which also provides a better objective for learning model parameters.
APA
Verma, P., Adam, V. & Solin, A.. (2024). Variational Gaussian Process Diffusion Processes. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:1909-1917 Available from https://proceedings.mlr.press/v238/verma24a.html.

Related Material