Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees

Zehong Wang, Zheyuan Zhang, Tianyi Ma, Nitesh V Chawla, Chuxu Zhang, Yanfang Ye
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:65518-65555, 2025.

Abstract

Foundation models are pretrained on large-scale corpora to learn generalizable patterns across domains and tasks—such as contours, textures, and edges in images, or tokens and sentences in text. In contrast, discovering such generalities in graph-structured data, especially across heterogeneous graph tasks, remains an open challenge. To address this, we propose a novel approach to cross-task generalization in graphs via task-trees, which serve as unified learning instances aligning node-, edge-, and graph-level tasks. We theoretically analyze the stability, transferability, and generalization properties of task-trees, showing that pretraining a graph neural network (GNN) on diverse task-trees with a reconstruction objective induces transferable knowledge. This enables efficient adaptation to downstream tasks with minimal fine-tuning. To validate our framework, we introduce Graph Generality Identifier on Task-Trees (GIT), a graph foundation model that demonstrates strong performance on over 30 graphs across five domains via fine-tuning, in-context learning, and zero-shot generalization. Code and data are available at https://github.com/Zehong-Wang/GIT.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-wang25eq, title = {Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees}, author = {Wang, Zehong and Zhang, Zheyuan and Ma, Tianyi and Chawla, Nitesh V and Zhang, Chuxu and Ye, Yanfang}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {65518--65555}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/wang25eq/wang25eq.pdf}, url = {https://proceedings.mlr.press/v267/wang25eq.html}, abstract = {Foundation models are pretrained on large-scale corpora to learn generalizable patterns across domains and tasks—such as contours, textures, and edges in images, or tokens and sentences in text. In contrast, discovering such generalities in graph-structured data, especially across heterogeneous graph tasks, remains an open challenge. To address this, we propose a novel approach to cross-task generalization in graphs via task-trees, which serve as unified learning instances aligning node-, edge-, and graph-level tasks. We theoretically analyze the stability, transferability, and generalization properties of task-trees, showing that pretraining a graph neural network (GNN) on diverse task-trees with a reconstruction objective induces transferable knowledge. This enables efficient adaptation to downstream tasks with minimal fine-tuning. To validate our framework, we introduce Graph Generality Identifier on Task-Trees (GIT), a graph foundation model that demonstrates strong performance on over 30 graphs across five domains via fine-tuning, in-context learning, and zero-shot generalization. Code and data are available at https://github.com/Zehong-Wang/GIT.} }
Endnote
%0 Conference Paper %T Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees %A Zehong Wang %A Zheyuan Zhang %A Tianyi Ma %A Nitesh V Chawla %A Chuxu Zhang %A Yanfang Ye %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-wang25eq %I PMLR %P 65518--65555 %U https://proceedings.mlr.press/v267/wang25eq.html %V 267 %X Foundation models are pretrained on large-scale corpora to learn generalizable patterns across domains and tasks—such as contours, textures, and edges in images, or tokens and sentences in text. In contrast, discovering such generalities in graph-structured data, especially across heterogeneous graph tasks, remains an open challenge. To address this, we propose a novel approach to cross-task generalization in graphs via task-trees, which serve as unified learning instances aligning node-, edge-, and graph-level tasks. We theoretically analyze the stability, transferability, and generalization properties of task-trees, showing that pretraining a graph neural network (GNN) on diverse task-trees with a reconstruction objective induces transferable knowledge. This enables efficient adaptation to downstream tasks with minimal fine-tuning. To validate our framework, we introduce Graph Generality Identifier on Task-Trees (GIT), a graph foundation model that demonstrates strong performance on over 30 graphs across five domains via fine-tuning, in-context learning, and zero-shot generalization. Code and data are available at https://github.com/Zehong-Wang/GIT.
APA
Wang, Z., Zhang, Z., Ma, T., Chawla, N.V., Zhang, C. & Ye, Y.. (2025). Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:65518-65555 Available from https://proceedings.mlr.press/v267/wang25eq.html.

Related Material