SubMix: Learning to Mix Graph Sampling Heuristics

Sami Abu-El-Haija, Joshua V. Dillon, Bahare Fatemi, Kyriakos Axiotis, Neslihan Bulut, Johannes Gasteiger, Bryan Perozzi, Mohammadhossein Bateni
Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, PMLR 216:1-10, 2023.

Abstract

Sampling subgraphs for training Graph Neural Networks (GNNs) is receiving much attention from the GNN community. While a variety of methods have been proposed, each method samples the graph according to its own heuristic. However, there has been little work in mixing these heuristics in an end-to-end trainable manner. In this work, we design a generative framework for graph sampling. Our method, SubMix, parameterizes subgraph sampling as a convex combination of heuristics. We show that a continuous relaxation of the discrete sampling process allows us to efficiently obtain analytical gradients for training the sampling parameters. Our experimental results illustrate the usefulness of learning graph sampling in three scenarios: (1) robust training of GNNs by automatically learning to discard noisy edge sources; (2) improving model performance by trainable and online edge subset selection; and (3) by integrating our framework into decoupled GNN models improves their performance on standard benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v216-abu-el-haija23a, title = {SubMix: Learning to Mix Graph Sampling Heuristics}, author = {Abu-El-Haija, Sami and Dillon, Joshua V. and Fatemi, Bahare and Axiotis, Kyriakos and Bulut, Neslihan and Gasteiger, Johannes and Perozzi, Bryan and Bateni, Mohammadhossein}, booktitle = {Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence}, pages = {1--10}, year = {2023}, editor = {Evans, Robin J. and Shpitser, Ilya}, volume = {216}, series = {Proceedings of Machine Learning Research}, month = {31 Jul--04 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v216/abu-el-haija23a/abu-el-haija23a.pdf}, url = {https://proceedings.mlr.press/v216/abu-el-haija23a.html}, abstract = {Sampling subgraphs for training Graph Neural Networks (GNNs) is receiving much attention from the GNN community. While a variety of methods have been proposed, each method samples the graph according to its own heuristic. However, there has been little work in mixing these heuristics in an end-to-end trainable manner. In this work, we design a generative framework for graph sampling. Our method, SubMix, parameterizes subgraph sampling as a convex combination of heuristics. We show that a continuous relaxation of the discrete sampling process allows us to efficiently obtain analytical gradients for training the sampling parameters. Our experimental results illustrate the usefulness of learning graph sampling in three scenarios: (1) robust training of GNNs by automatically learning to discard noisy edge sources; (2) improving model performance by trainable and online edge subset selection; and (3) by integrating our framework into decoupled GNN models improves their performance on standard benchmarks.} }
Endnote
%0 Conference Paper %T SubMix: Learning to Mix Graph Sampling Heuristics %A Sami Abu-El-Haija %A Joshua V. Dillon %A Bahare Fatemi %A Kyriakos Axiotis %A Neslihan Bulut %A Johannes Gasteiger %A Bryan Perozzi %A Mohammadhossein Bateni %B Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2023 %E Robin J. Evans %E Ilya Shpitser %F pmlr-v216-abu-el-haija23a %I PMLR %P 1--10 %U https://proceedings.mlr.press/v216/abu-el-haija23a.html %V 216 %X Sampling subgraphs for training Graph Neural Networks (GNNs) is receiving much attention from the GNN community. While a variety of methods have been proposed, each method samples the graph according to its own heuristic. However, there has been little work in mixing these heuristics in an end-to-end trainable manner. In this work, we design a generative framework for graph sampling. Our method, SubMix, parameterizes subgraph sampling as a convex combination of heuristics. We show that a continuous relaxation of the discrete sampling process allows us to efficiently obtain analytical gradients for training the sampling parameters. Our experimental results illustrate the usefulness of learning graph sampling in three scenarios: (1) robust training of GNNs by automatically learning to discard noisy edge sources; (2) improving model performance by trainable and online edge subset selection; and (3) by integrating our framework into decoupled GNN models improves their performance on standard benchmarks.
APA
Abu-El-Haija, S., Dillon, J.V., Fatemi, B., Axiotis, K., Bulut, N., Gasteiger, J., Perozzi, B. & Bateni, M.. (2023). SubMix: Learning to Mix Graph Sampling Heuristics. Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 216:1-10 Available from https://proceedings.mlr.press/v216/abu-el-haija23a.html.

Related Material