LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification

Hang Gao, Huang Wenxuan, Fengge Wu, Zhao Junsuo, Changwen Zheng, Huaping Liu
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:18585-18611, 2025.

Abstract

The use of large language models (LLMs) as feature enhancers to optimize node representations, which are then used as inputs for graph neural networks (GNNs), has shown significant potential in graph representation learning. However, the fundamental properties of this approach remain underexplored. To address this issue, we propose conducting a more in-depth analysis of this issue based on the interchange intervention method. First, we construct a synthetic graph dataset with controllable causal relationships, enabling precise manipulation of semantic relationships and causal modeling to provide data for analysis. Using this dataset, we conduct interchange interventions to examine the deeper properties of LLM enhancers and GNNs, uncovering their underlying logic and internal mechanisms. Building on the analytical results, we design a plug-and-play optimization module to improve the information transfer between LLM enhancers and GNNs. Experiments across multiple datasets and models validate the proposed module.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-gao25o, title = {{LLM} Enhancers for {GNN}s: An Analysis from the Perspective of Causal Mechanism Identification}, author = {Gao, Hang and Wenxuan, Huang and Wu, Fengge and Junsuo, Zhao and Zheng, Changwen and Liu, Huaping}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {18585--18611}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/gao25o/gao25o.pdf}, url = {https://proceedings.mlr.press/v267/gao25o.html}, abstract = {The use of large language models (LLMs) as feature enhancers to optimize node representations, which are then used as inputs for graph neural networks (GNNs), has shown significant potential in graph representation learning. However, the fundamental properties of this approach remain underexplored. To address this issue, we propose conducting a more in-depth analysis of this issue based on the interchange intervention method. First, we construct a synthetic graph dataset with controllable causal relationships, enabling precise manipulation of semantic relationships and causal modeling to provide data for analysis. Using this dataset, we conduct interchange interventions to examine the deeper properties of LLM enhancers and GNNs, uncovering their underlying logic and internal mechanisms. Building on the analytical results, we design a plug-and-play optimization module to improve the information transfer between LLM enhancers and GNNs. Experiments across multiple datasets and models validate the proposed module.} }
Endnote
%0 Conference Paper %T LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification %A Hang Gao %A Huang Wenxuan %A Fengge Wu %A Zhao Junsuo %A Changwen Zheng %A Huaping Liu %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-gao25o %I PMLR %P 18585--18611 %U https://proceedings.mlr.press/v267/gao25o.html %V 267 %X The use of large language models (LLMs) as feature enhancers to optimize node representations, which are then used as inputs for graph neural networks (GNNs), has shown significant potential in graph representation learning. However, the fundamental properties of this approach remain underexplored. To address this issue, we propose conducting a more in-depth analysis of this issue based on the interchange intervention method. First, we construct a synthetic graph dataset with controllable causal relationships, enabling precise manipulation of semantic relationships and causal modeling to provide data for analysis. Using this dataset, we conduct interchange interventions to examine the deeper properties of LLM enhancers and GNNs, uncovering their underlying logic and internal mechanisms. Building on the analytical results, we design a plug-and-play optimization module to improve the information transfer between LLM enhancers and GNNs. Experiments across multiple datasets and models validate the proposed module.
APA
Gao, H., Wenxuan, H., Wu, F., Junsuo, Z., Zheng, C. & Liu, H.. (2025). LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:18585-18611 Available from https://proceedings.mlr.press/v267/gao25o.html.

Related Material