A New Perspective on the Effects of Spectrum in Graph Neural Networks

Mingqi Yang, Yanming Shen, Rui Li, Heng Qi, Qiang Zhang, Baocai Yin
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:25261-25279, 2022.

Abstract

Many improvements on GNNs can be deemed as operations on the spectrum of the underlying graph matrix, which motivates us to directly study the characteristics of the spectrum and their effects on GNN performance. By generalizing most existing GNN architectures, we show that the correlation issue caused by the unsmooth spectrum becomes the obstacle to leveraging more powerful graph filters as well as developing deep architectures, which therefore restricts GNNs’ performance. Inspired by this, we propose the correlation-free architecture which naturally removes the correlation issue among different channels, making it possible to utilize more sophisticated filters within each channel. The final correlation-free architecture with more powerful filters consistently boosts the performance of learning graph representations. Code is available at https://github.com/qslim/gnn-spectrum.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-yang22n, title = {A New Perspective on the Effects of Spectrum in Graph Neural Networks}, author = {Yang, Mingqi and Shen, Yanming and Li, Rui and Qi, Heng and Zhang, Qiang and Yin, Baocai}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {25261--25279}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/yang22n/yang22n.pdf}, url = {https://proceedings.mlr.press/v162/yang22n.html}, abstract = {Many improvements on GNNs can be deemed as operations on the spectrum of the underlying graph matrix, which motivates us to directly study the characteristics of the spectrum and their effects on GNN performance. By generalizing most existing GNN architectures, we show that the correlation issue caused by the unsmooth spectrum becomes the obstacle to leveraging more powerful graph filters as well as developing deep architectures, which therefore restricts GNNs’ performance. Inspired by this, we propose the correlation-free architecture which naturally removes the correlation issue among different channels, making it possible to utilize more sophisticated filters within each channel. The final correlation-free architecture with more powerful filters consistently boosts the performance of learning graph representations. Code is available at https://github.com/qslim/gnn-spectrum.} }
Endnote
%0 Conference Paper %T A New Perspective on the Effects of Spectrum in Graph Neural Networks %A Mingqi Yang %A Yanming Shen %A Rui Li %A Heng Qi %A Qiang Zhang %A Baocai Yin %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-yang22n %I PMLR %P 25261--25279 %U https://proceedings.mlr.press/v162/yang22n.html %V 162 %X Many improvements on GNNs can be deemed as operations on the spectrum of the underlying graph matrix, which motivates us to directly study the characteristics of the spectrum and their effects on GNN performance. By generalizing most existing GNN architectures, we show that the correlation issue caused by the unsmooth spectrum becomes the obstacle to leveraging more powerful graph filters as well as developing deep architectures, which therefore restricts GNNs’ performance. Inspired by this, we propose the correlation-free architecture which naturally removes the correlation issue among different channels, making it possible to utilize more sophisticated filters within each channel. The final correlation-free architecture with more powerful filters consistently boosts the performance of learning graph representations. Code is available at https://github.com/qslim/gnn-spectrum.
APA
Yang, M., Shen, Y., Li, R., Qi, H., Zhang, Q. & Yin, B.. (2022). A New Perspective on the Effects of Spectrum in Graph Neural Networks. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:25261-25279 Available from https://proceedings.mlr.press/v162/yang22n.html.

Related Material