SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data

Maud Lemercier, Cristopher Salvi, Thomas Cass, Edwin V. Bonilla, Theodoros Damoulas, Terry J Lyons
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:6233-6242, 2021.

Abstract

Making predictions and quantifying their uncertainty when the input data is sequential is a fundamental learning challenge, recently attracting increasing attention. We develop SigGPDE, a new scalable sparse variational inference framework for Gaussian Processes (GPs) on sequential data. Our contribution is twofold. First, we construct inducing variables underpinning the sparse approximation so that the resulting evidence lower bound (ELBO) does not require any matrix inversion. Second, we show that the gradients of the GP signature kernel are solutions of a hyperbolic partial differential equation (PDE). This theoretical insight allows us to build an efficient back-propagation algorithm to optimize the ELBO. We showcase the significant computational gains of SigGPDE compared to existing methods, while achieving state-of-the-art performance for classification tasks on large datasets of up to 1 million multivariate time series.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-lemercier21a, title = {SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data}, author = {Lemercier, Maud and Salvi, Cristopher and Cass, Thomas and Bonilla, Edwin V. and Damoulas, Theodoros and Lyons, Terry J}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {6233--6242}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/lemercier21a/lemercier21a.pdf}, url = {https://proceedings.mlr.press/v139/lemercier21a.html}, abstract = {Making predictions and quantifying their uncertainty when the input data is sequential is a fundamental learning challenge, recently attracting increasing attention. We develop SigGPDE, a new scalable sparse variational inference framework for Gaussian Processes (GPs) on sequential data. Our contribution is twofold. First, we construct inducing variables underpinning the sparse approximation so that the resulting evidence lower bound (ELBO) does not require any matrix inversion. Second, we show that the gradients of the GP signature kernel are solutions of a hyperbolic partial differential equation (PDE). This theoretical insight allows us to build an efficient back-propagation algorithm to optimize the ELBO. We showcase the significant computational gains of SigGPDE compared to existing methods, while achieving state-of-the-art performance for classification tasks on large datasets of up to 1 million multivariate time series.} }
Endnote
%0 Conference Paper %T SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data %A Maud Lemercier %A Cristopher Salvi %A Thomas Cass %A Edwin V. Bonilla %A Theodoros Damoulas %A Terry J Lyons %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-lemercier21a %I PMLR %P 6233--6242 %U https://proceedings.mlr.press/v139/lemercier21a.html %V 139 %X Making predictions and quantifying their uncertainty when the input data is sequential is a fundamental learning challenge, recently attracting increasing attention. We develop SigGPDE, a new scalable sparse variational inference framework for Gaussian Processes (GPs) on sequential data. Our contribution is twofold. First, we construct inducing variables underpinning the sparse approximation so that the resulting evidence lower bound (ELBO) does not require any matrix inversion. Second, we show that the gradients of the GP signature kernel are solutions of a hyperbolic partial differential equation (PDE). This theoretical insight allows us to build an efficient back-propagation algorithm to optimize the ELBO. We showcase the significant computational gains of SigGPDE compared to existing methods, while achieving state-of-the-art performance for classification tasks on large datasets of up to 1 million multivariate time series.
APA
Lemercier, M., Salvi, C., Cass, T., Bonilla, E.V., Damoulas, T. & Lyons, T.J.. (2021). SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:6233-6242 Available from https://proceedings.mlr.press/v139/lemercier21a.html.

Related Material