Learning in Deep Factor Graphs with Gaussian Belief Propagation

Seth Nabarro, Mark Van Der Wilk, Andrew Davison
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:37141-37163, 2024.

Abstract

We propose an approach to do learning in Gaussian factor graphs. We treat all relevant quantities (inputs, outputs, parameters, activations) as random variables in a graphical model, and view training and prediction as inference problems with different observed nodes. Our experiments show that these problems can be efficiently solved with belief propagation (BP), whose updates are inherently local, presenting exciting opportunities for distributed and asynchronous training. Our approach can be scaled to deep networks and provides a natural means to do continual learning: use the BP-estimated posterior of the current task as a prior for the next. On a video denoising task we demonstrate the benefit of learnable parameters over a classical factor graph approach and we show encouraging performance of deep factor graphs for continual image classification.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-nabarro24a, title = {Learning in Deep Factor Graphs with {G}aussian Belief Propagation}, author = {Nabarro, Seth and Van Der Wilk, Mark and Davison, Andrew}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {37141--37163}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/nabarro24a/nabarro24a.pdf}, url = {https://proceedings.mlr.press/v235/nabarro24a.html}, abstract = {We propose an approach to do learning in Gaussian factor graphs. We treat all relevant quantities (inputs, outputs, parameters, activations) as random variables in a graphical model, and view training and prediction as inference problems with different observed nodes. Our experiments show that these problems can be efficiently solved with belief propagation (BP), whose updates are inherently local, presenting exciting opportunities for distributed and asynchronous training. Our approach can be scaled to deep networks and provides a natural means to do continual learning: use the BP-estimated posterior of the current task as a prior for the next. On a video denoising task we demonstrate the benefit of learnable parameters over a classical factor graph approach and we show encouraging performance of deep factor graphs for continual image classification.} }
Endnote
%0 Conference Paper %T Learning in Deep Factor Graphs with Gaussian Belief Propagation %A Seth Nabarro %A Mark Van Der Wilk %A Andrew Davison %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-nabarro24a %I PMLR %P 37141--37163 %U https://proceedings.mlr.press/v235/nabarro24a.html %V 235 %X We propose an approach to do learning in Gaussian factor graphs. We treat all relevant quantities (inputs, outputs, parameters, activations) as random variables in a graphical model, and view training and prediction as inference problems with different observed nodes. Our experiments show that these problems can be efficiently solved with belief propagation (BP), whose updates are inherently local, presenting exciting opportunities for distributed and asynchronous training. Our approach can be scaled to deep networks and provides a natural means to do continual learning: use the BP-estimated posterior of the current task as a prior for the next. On a video denoising task we demonstrate the benefit of learnable parameters over a classical factor graph approach and we show encouraging performance of deep factor graphs for continual image classification.
APA
Nabarro, S., Van Der Wilk, M. & Davison, A.. (2024). Learning in Deep Factor Graphs with Gaussian Belief Propagation. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:37141-37163 Available from https://proceedings.mlr.press/v235/nabarro24a.html.

Related Material