Adaptive RKHS Fourier Features for Compositional Gaussian Process Models

Xinxing Shi, Thomas Baldwin-McDonald, Mauricio A Álvarez
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:1738-1746, 2025.

Abstract

Deep Gaussian Processes (DGPs) leverage a compositional structure to model non-stationary processes. DGPs typically rely on local inducing point approximations across intermediate GP layers. Recent advances in DGP inference have shown that incorporating global Fourier features from the Reproducing Kernel Hilbert Space (RKHS) can enhance the DGPs’ capability to capture complex non-stationary patterns. This paper extends the use of these features to compositional GPs involving linear transformations. In particular, we introduce Ordinary Differential Equation(ODE)–based RKHS Fourier features that allow for adaptive amplitude and phase modulation through convolution operations. This convolutional formulation relates our work to recently proposed deep latent force models, a multi-layer structure designed for modelling nonlinear dynamical systems. By embedding these adjustable RKHS Fourier features within a doubly stochastic variational inference framework, our model exhibits improved predictive performance across various regression tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-shi25b, title = {Adaptive RKHS Fourier Features for Compositional Gaussian Process Models}, author = {Shi, Xinxing and Baldwin-McDonald, Thomas and {\'A}lvarez, Mauricio A}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {1738--1746}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/shi25b/shi25b.pdf}, url = {https://proceedings.mlr.press/v258/shi25b.html}, abstract = {Deep Gaussian Processes (DGPs) leverage a compositional structure to model non-stationary processes. DGPs typically rely on local inducing point approximations across intermediate GP layers. Recent advances in DGP inference have shown that incorporating global Fourier features from the Reproducing Kernel Hilbert Space (RKHS) can enhance the DGPs’ capability to capture complex non-stationary patterns. This paper extends the use of these features to compositional GPs involving linear transformations. In particular, we introduce Ordinary Differential Equation(ODE)–based RKHS Fourier features that allow for adaptive amplitude and phase modulation through convolution operations. This convolutional formulation relates our work to recently proposed deep latent force models, a multi-layer structure designed for modelling nonlinear dynamical systems. By embedding these adjustable RKHS Fourier features within a doubly stochastic variational inference framework, our model exhibits improved predictive performance across various regression tasks.} }
Endnote
%0 Conference Paper %T Adaptive RKHS Fourier Features for Compositional Gaussian Process Models %A Xinxing Shi %A Thomas Baldwin-McDonald %A Mauricio A Álvarez %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-shi25b %I PMLR %P 1738--1746 %U https://proceedings.mlr.press/v258/shi25b.html %V 258 %X Deep Gaussian Processes (DGPs) leverage a compositional structure to model non-stationary processes. DGPs typically rely on local inducing point approximations across intermediate GP layers. Recent advances in DGP inference have shown that incorporating global Fourier features from the Reproducing Kernel Hilbert Space (RKHS) can enhance the DGPs’ capability to capture complex non-stationary patterns. This paper extends the use of these features to compositional GPs involving linear transformations. In particular, we introduce Ordinary Differential Equation(ODE)–based RKHS Fourier features that allow for adaptive amplitude and phase modulation through convolution operations. This convolutional formulation relates our work to recently proposed deep latent force models, a multi-layer structure designed for modelling nonlinear dynamical systems. By embedding these adjustable RKHS Fourier features within a doubly stochastic variational inference framework, our model exhibits improved predictive performance across various regression tasks.
APA
Shi, X., Baldwin-McDonald, T. & Álvarez, M.A.. (2025). Adaptive RKHS Fourier Features for Compositional Gaussian Process Models. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:1738-1746 Available from https://proceedings.mlr.press/v258/shi25b.html.

Related Material