Monotone Operator Theory-Inspired Message Passing for Learning Long-Range Interaction on Graphs

Justin M. Baker, Qingsong Wang, Martin Berzins, Thomas Strohmer, Bao Wang
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:2233-2241, 2024.

Abstract

Learning long-range interactions (LRI) between distant nodes is crucial for many graph learning tasks. Predominant graph neural networks (GNNs) rely on local message passing and struggle to learn LRI. In this paper, we propose DRGNN to learn LRI leveraging monotone operator theory. DRGNN contains two key components: (1) we use a full node similarity matrix beyond adjacency matrix – drawing inspiration from the personalized PageRank matrix – as the aggregation matrix for message passing, and (2) we implement message-passing on graphs using Douglas-Rachford splitting to circumvent prohibitive matrix inversion. We demonstrate that DRGNN surpasses various advanced GNNs, including Transformer-based models, on several benchmark LRI learning tasks arising from different application domains, highlighting its efficacy in learning LRI. Code is available at \url{https://github.com/Utah-Math-Data-Science/PR-inspired-aggregation}.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-baker24a, title = {Monotone Operator Theory-Inspired Message Passing for Learning Long-Range Interaction on Graphs}, author = {Baker, Justin M. and Wang, Qingsong and Berzins, Martin and Strohmer, Thomas and Wang, Bao}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {2233--2241}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/baker24a/baker24a.pdf}, url = {https://proceedings.mlr.press/v238/baker24a.html}, abstract = {Learning long-range interactions (LRI) between distant nodes is crucial for many graph learning tasks. Predominant graph neural networks (GNNs) rely on local message passing and struggle to learn LRI. In this paper, we propose DRGNN to learn LRI leveraging monotone operator theory. DRGNN contains two key components: (1) we use a full node similarity matrix beyond adjacency matrix – drawing inspiration from the personalized PageRank matrix – as the aggregation matrix for message passing, and (2) we implement message-passing on graphs using Douglas-Rachford splitting to circumvent prohibitive matrix inversion. We demonstrate that DRGNN surpasses various advanced GNNs, including Transformer-based models, on several benchmark LRI learning tasks arising from different application domains, highlighting its efficacy in learning LRI. Code is available at \url{https://github.com/Utah-Math-Data-Science/PR-inspired-aggregation}.} }
Endnote
%0 Conference Paper %T Monotone Operator Theory-Inspired Message Passing for Learning Long-Range Interaction on Graphs %A Justin M. Baker %A Qingsong Wang %A Martin Berzins %A Thomas Strohmer %A Bao Wang %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-baker24a %I PMLR %P 2233--2241 %U https://proceedings.mlr.press/v238/baker24a.html %V 238 %X Learning long-range interactions (LRI) between distant nodes is crucial for many graph learning tasks. Predominant graph neural networks (GNNs) rely on local message passing and struggle to learn LRI. In this paper, we propose DRGNN to learn LRI leveraging monotone operator theory. DRGNN contains two key components: (1) we use a full node similarity matrix beyond adjacency matrix – drawing inspiration from the personalized PageRank matrix – as the aggregation matrix for message passing, and (2) we implement message-passing on graphs using Douglas-Rachford splitting to circumvent prohibitive matrix inversion. We demonstrate that DRGNN surpasses various advanced GNNs, including Transformer-based models, on several benchmark LRI learning tasks arising from different application domains, highlighting its efficacy in learning LRI. Code is available at \url{https://github.com/Utah-Math-Data-Science/PR-inspired-aggregation}.
APA
Baker, J.M., Wang, Q., Berzins, M., Strohmer, T. & Wang, B.. (2024). Monotone Operator Theory-Inspired Message Passing for Learning Long-Range Interaction on Graphs. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:2233-2241 Available from https://proceedings.mlr.press/v238/baker24a.html.

Related Material