Do We Really Need Complicated Graph Learning Models? – A Simple but Effective Baseline

Kaan Sancak, Muhammed Fatih Balin, Umit Catalyurek
Proceedings of the Third Learning on Graphs Conference, PMLR 269:23:1-23:19, 2025.

Abstract

Despite advances in graph learning, increasingly complex models introduce significant overheads, including prolonged preprocessing and training times, excessive memory requirements, and numerous hyperparameters which often limit their scalability to large datasets. Consequently, evaluating model effectiveness in this rapidly growing field has become increasingly challenging. We investigate whether complicated methods are necessary if foundational and scalable models can achieve better quality on large datasets. We first demonstrate that Graph Convolutional Network (GCN) is able to achieve competitive quality using skip connections on large datasets. Next, we argue that existing Graph Neural Network (GNN) skip connections are incomplete, lacking neighborhood embeddings within them. To address this, we introduce Neighbor Aware Skip Connections (NASC), a novel skip connection with an adaptive weighting strategy. Our evaluation show that GCN with NASC outperforms various baselines on large datasets, including GNNs and Graph Transformers (GTs), with negligible overheads, which we analyze both theoretically and empirically. We also demonstrate that NASC can be integrated into GTs, boosting performance across over 10 benchmark datasets with various properties and tasks. NASC empowers researchers to establish a robust baseline performance for large datasets, eliminating the need for extensive hyperparameter tuning, while supporting mini-batch training and seamless integration with popular graph learning libraries.

Cite this Paper

BibTeX
@InProceedings{pmlr-v269-sancak25a, title = {Do We Really Need Complicated Graph Learning Models? – A Simple but Effective Baseline}, author = {Sancak, Kaan and Balin, Muhammed Fatih and Catalyurek, Umit}, booktitle = {Proceedings of the Third Learning on Graphs Conference}, pages = {23:1--23:19}, year = {2025}, editor = {Wolf, Guy and Krishnaswamy, Smita}, volume = {269}, series = {Proceedings of Machine Learning Research}, month = {26--29 Nov}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v269/main/assets/sancak25a/sancak25a.pdf}, url = {https://proceedings.mlr.press/v269/sancak25a.html}, abstract = {Despite advances in graph learning, increasingly complex models introduce significant overheads, including prolonged preprocessing and training times, excessive memory requirements, and numerous hyperparameters which often limit their scalability to large datasets. Consequently, evaluating model effectiveness in this rapidly growing field has become increasingly challenging. We investigate whether complicated methods are necessary if foundational and scalable models can achieve better quality on large datasets. We first demonstrate that Graph Convolutional Network (GCN) is able to achieve competitive quality using skip connections on large datasets. Next, we argue that existing Graph Neural Network (GNN) skip connections are incomplete, lacking neighborhood embeddings within them. To address this, we introduce Neighbor Aware Skip Connections (NASC), a novel skip connection with an adaptive weighting strategy. Our evaluation show that GCN with NASC outperforms various baselines on large datasets, including GNNs and Graph Transformers (GTs), with negligible overheads, which we analyze both theoretically and empirically. We also demonstrate that NASC can be integrated into GTs, boosting performance across over 10 benchmark datasets with various properties and tasks. NASC empowers researchers to establish a robust baseline performance for large datasets, eliminating the need for extensive hyperparameter tuning, while supporting mini-batch training and seamless integration with popular graph learning libraries.} }
Endnote
%0 Conference Paper %T Do We Really Need Complicated Graph Learning Models? – A Simple but Effective Baseline %A Kaan Sancak %A Muhammed Fatih Balin %A Umit Catalyurek %B Proceedings of the Third Learning on Graphs Conference %C Proceedings of Machine Learning Research %D 2025 %E Guy Wolf %E Smita Krishnaswamy %F pmlr-v269-sancak25a %I PMLR %P 23:1--23:19 %U https://proceedings.mlr.press/v269/sancak25a.html %V 269 %X Despite advances in graph learning, increasingly complex models introduce significant overheads, including prolonged preprocessing and training times, excessive memory requirements, and numerous hyperparameters which often limit their scalability to large datasets. Consequently, evaluating model effectiveness in this rapidly growing field has become increasingly challenging. We investigate whether complicated methods are necessary if foundational and scalable models can achieve better quality on large datasets. We first demonstrate that Graph Convolutional Network (GCN) is able to achieve competitive quality using skip connections on large datasets. Next, we argue that existing Graph Neural Network (GNN) skip connections are incomplete, lacking neighborhood embeddings within them. To address this, we introduce Neighbor Aware Skip Connections (NASC), a novel skip connection with an adaptive weighting strategy. Our evaluation show that GCN with NASC outperforms various baselines on large datasets, including GNNs and Graph Transformers (GTs), with negligible overheads, which we analyze both theoretically and empirically. We also demonstrate that NASC can be integrated into GTs, boosting performance across over 10 benchmark datasets with various properties and tasks. NASC empowers researchers to establish a robust baseline performance for large datasets, eliminating the need for extensive hyperparameter tuning, while supporting mini-batch training and seamless integration with popular graph learning libraries.
APA
Sancak, K., Balin, M.F. & Catalyurek, U.. (2025). Do We Really Need Complicated Graph Learning Models? – A Simple but Effective Baseline. Proceedings of the Third Learning on Graphs Conference, in Proceedings of Machine Learning Research 269:23:1-23:19 Available from https://proceedings.mlr.press/v269/sancak25a.html.

Related Material