Scalable Deep Gaussian Markov Random Fields for General Graphs

Joel Oskarsson, Per Sidén, Fredrik Lindsten
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:17117-17137, 2022.

Abstract

Machine learning methods on graphs have proven useful in many applications due to their ability to handle generally structured data. The framework of Gaussian Markov Random Fields (GMRFs) provides a principled way to define Gaussian models on graphs by utilizing their sparsity structure. We propose a flexible GMRF model for general graphs built on the multi-layer structure of Deep GMRFs, originally proposed for lattice graphs only. By designing a new type of layer we enable the model to scale to large graphs. The layer is constructed to allow for efficient training using variational inference and existing software frameworks for Graph Neural Networks. For a Gaussian likelihood, close to exact Bayesian inference is available for the latent field. This allows for making predictions with accompanying uncertainty estimates. The usefulness of the proposed model is verified by experiments on a number of synthetic and real world datasets, where it compares favorably to other both Bayesian and deep learning methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-oskarsson22a, title = {Scalable Deep {G}aussian {M}arkov Random Fields for General Graphs}, author = {Oskarsson, Joel and Sid{\'e}n, Per and Lindsten, Fredrik}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {17117--17137}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/oskarsson22a/oskarsson22a.pdf}, url = {https://proceedings.mlr.press/v162/oskarsson22a.html}, abstract = {Machine learning methods on graphs have proven useful in many applications due to their ability to handle generally structured data. The framework of Gaussian Markov Random Fields (GMRFs) provides a principled way to define Gaussian models on graphs by utilizing their sparsity structure. We propose a flexible GMRF model for general graphs built on the multi-layer structure of Deep GMRFs, originally proposed for lattice graphs only. By designing a new type of layer we enable the model to scale to large graphs. The layer is constructed to allow for efficient training using variational inference and existing software frameworks for Graph Neural Networks. For a Gaussian likelihood, close to exact Bayesian inference is available for the latent field. This allows for making predictions with accompanying uncertainty estimates. The usefulness of the proposed model is verified by experiments on a number of synthetic and real world datasets, where it compares favorably to other both Bayesian and deep learning methods.} }
Endnote
%0 Conference Paper %T Scalable Deep Gaussian Markov Random Fields for General Graphs %A Joel Oskarsson %A Per Sidén %A Fredrik Lindsten %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-oskarsson22a %I PMLR %P 17117--17137 %U https://proceedings.mlr.press/v162/oskarsson22a.html %V 162 %X Machine learning methods on graphs have proven useful in many applications due to their ability to handle generally structured data. The framework of Gaussian Markov Random Fields (GMRFs) provides a principled way to define Gaussian models on graphs by utilizing their sparsity structure. We propose a flexible GMRF model for general graphs built on the multi-layer structure of Deep GMRFs, originally proposed for lattice graphs only. By designing a new type of layer we enable the model to scale to large graphs. The layer is constructed to allow for efficient training using variational inference and existing software frameworks for Graph Neural Networks. For a Gaussian likelihood, close to exact Bayesian inference is available for the latent field. This allows for making predictions with accompanying uncertainty estimates. The usefulness of the proposed model is verified by experiments on a number of synthetic and real world datasets, where it compares favorably to other both Bayesian and deep learning methods.
APA
Oskarsson, J., Sidén, P. & Lindsten, F.. (2022). Scalable Deep Gaussian Markov Random Fields for General Graphs. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:17117-17137 Available from https://proceedings.mlr.press/v162/oskarsson22a.html.

Related Material