Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era

Nicolas Durrande, Vincent Adam, Lucas Bordeaux, Stefanos Eleftheriadis, James Hensman
Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, PMLR 89:2780-2789, 2019.

Abstract

Banded matrices can be used as precision matrices in several models including linear state-space models, some Gaussian processes, and Gaussian Markov random fields. The aim of the paper is to make modern inference methods (such as variational inference or gradient-based sampling) available for Gaussian models with banded precision. We show that this can efficiently be achieved by equipping an automatic differentiation framework, such as TensorFlow or PyTorch, with some linear algebra operators dedicated to banded matrices. This paper studies the algorithmic aspects of the required operators, details their reverse-mode derivatives, and show that their complexity is linear in the number of observations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v89-durrande19a, title = {Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era}, author = {Durrande, Nicolas and Adam, Vincent and Bordeaux, Lucas and Eleftheriadis, Stefanos and Hensman, James}, booktitle = {Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics}, pages = {2780--2789}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/durrande19a/durrande19a.pdf}, url = {https://proceedings.mlr.press/v89/durrande19a.html}, abstract = {Banded matrices can be used as precision matrices in several models including linear state-space models, some Gaussian processes, and Gaussian Markov random fields. The aim of the paper is to make modern inference methods (such as variational inference or gradient-based sampling) available for Gaussian models with banded precision. We show that this can efficiently be achieved by equipping an automatic differentiation framework, such as TensorFlow or PyTorch, with some linear algebra operators dedicated to banded matrices. This paper studies the algorithmic aspects of the required operators, details their reverse-mode derivatives, and show that their complexity is linear in the number of observations.} }
Endnote
%0 Conference Paper %T Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era %A Nicolas Durrande %A Vincent Adam %A Lucas Bordeaux %A Stefanos Eleftheriadis %A James Hensman %B Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Masashi Sugiyama %F pmlr-v89-durrande19a %I PMLR %P 2780--2789 %U https://proceedings.mlr.press/v89/durrande19a.html %V 89 %X Banded matrices can be used as precision matrices in several models including linear state-space models, some Gaussian processes, and Gaussian Markov random fields. The aim of the paper is to make modern inference methods (such as variational inference or gradient-based sampling) available for Gaussian models with banded precision. We show that this can efficiently be achieved by equipping an automatic differentiation framework, such as TensorFlow or PyTorch, with some linear algebra operators dedicated to banded matrices. This paper studies the algorithmic aspects of the required operators, details their reverse-mode derivatives, and show that their complexity is linear in the number of observations.
APA
Durrande, N., Adam, V., Bordeaux, L., Eleftheriadis, S. & Hensman, J.. (2019). Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era. Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 89:2780-2789 Available from https://proceedings.mlr.press/v89/durrande19a.html.

Related Material