Consensus Optimization Graph Neural Networks

Olga Solodova, Ryan P Adams
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:6445-6466, 2026.

Abstract

Implicit graph neural networks (GNNs) cast inference as optimization, computing node embeddings as the solution to a graph-structured optimization problem. By coupling node embeddings through a shared global objective, implicit GNNs naturally capture long-range dependencies, addressing a key limitation of standard message-passing GNNs. Another appealing property of these models is that inference can be performed in a decentralized and asynchronous manner, making them well suited for deployment in multi-agent systems. However, training is costly, as it requires solving optimization problems in both the forward and backward pass. In this work, we focus on graph-level prediction and introduce an approach for directly computing graph-level representations in implicit GNNs that bypasses the need for node embeddings. We do this by reformulating the internal optimization problem to operate over a shared graph embedding. This significantly reduces both computation and memory costs, while retaining compatibility with decentralized and asynchronous inference. Experiments across multiple benchmarks show that the shared-embedding approach consistently matches the predictive performance of using node embeddings, providing a simpler and more efficient alternative for graph-level tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-solodova26a, title = {Consensus Optimization Graph Neural Networks}, author = {Solodova, Olga and Adams, Ryan P}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {6445--6466}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/solodova26a/solodova26a.pdf}, url = {https://proceedings.mlr.press/v337/solodova26a.html}, abstract = {Implicit graph neural networks (GNNs) cast inference as optimization, computing node embeddings as the solution to a graph-structured optimization problem. By coupling node embeddings through a shared global objective, implicit GNNs naturally capture long-range dependencies, addressing a key limitation of standard message-passing GNNs. Another appealing property of these models is that inference can be performed in a decentralized and asynchronous manner, making them well suited for deployment in multi-agent systems. However, training is costly, as it requires solving optimization problems in both the forward and backward pass. In this work, we focus on graph-level prediction and introduce an approach for directly computing graph-level representations in implicit GNNs that bypasses the need for node embeddings. We do this by reformulating the internal optimization problem to operate over a shared graph embedding. This significantly reduces both computation and memory costs, while retaining compatibility with decentralized and asynchronous inference. Experiments across multiple benchmarks show that the shared-embedding approach consistently matches the predictive performance of using node embeddings, providing a simpler and more efficient alternative for graph-level tasks.} }
Endnote
%0 Conference Paper %T Consensus Optimization Graph Neural Networks %A Olga Solodova %A Ryan P Adams %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-solodova26a %I PMLR %P 6445--6466 %U https://proceedings.mlr.press/v337/solodova26a.html %V 337 %X Implicit graph neural networks (GNNs) cast inference as optimization, computing node embeddings as the solution to a graph-structured optimization problem. By coupling node embeddings through a shared global objective, implicit GNNs naturally capture long-range dependencies, addressing a key limitation of standard message-passing GNNs. Another appealing property of these models is that inference can be performed in a decentralized and asynchronous manner, making them well suited for deployment in multi-agent systems. However, training is costly, as it requires solving optimization problems in both the forward and backward pass. In this work, we focus on graph-level prediction and introduce an approach for directly computing graph-level representations in implicit GNNs that bypasses the need for node embeddings. We do this by reformulating the internal optimization problem to operate over a shared graph embedding. This significantly reduces both computation and memory costs, while retaining compatibility with decentralized and asynchronous inference. Experiments across multiple benchmarks show that the shared-embedding approach consistently matches the predictive performance of using node embeddings, providing a simpler and more efficient alternative for graph-level tasks.
APA
Solodova, O. & Adams, R.P.. (2026). Consensus Optimization Graph Neural Networks. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:6445-6466 Available from https://proceedings.mlr.press/v337/solodova26a.html.

Related Material