Graph Contrastive Learning with Group Whitening

Chunhui Zhang, Rui Miao
Proceedings of the 15th Asian Conference on Machine Learning, PMLR 222:1622-1637, 2024.

Abstract

Graph neural networks (GNNs) have demonstrated their great power in learning graph-structured data. Due to the limitations of expensive labeled data, contrastive learning has been applied in graph domain. We propose GWGCL, a graph contrastive learning method based on feature group whitening to achieve two key properties of contrastive learning: alignment and uniformity. GWGCL achieves the alignment by ensuring consistency between positive samples. There is no need for negative samples to participate, but rather to achieve the uniformity between samples through whitening. Because whitening has the effect of feature divergence, it avoids the collapse of all sample representations to a single point, which is called dimensional collapse. Moreover, GWGCL can achieve better results and higher efficiency without the need for asymmetric networks, projection layers, stopping gradients and complex loss function. Through extensive experiments, GWGCL performs competitively on node classification and graph classification tasks across ten common graph datasets.The code is in: https://github.com/MR9812/GWGCL.

Cite this Paper

BibTeX
@InProceedings{pmlr-v222-zhang24a, title = {Graph Contrastive Learning with Group Whitening}, author = {Zhang, Chunhui and Miao, Rui}, booktitle = {Proceedings of the 15th Asian Conference on Machine Learning}, pages = {1622--1637}, year = {2024}, editor = {Yanıkoğlu, Berrin and Buntine, Wray}, volume = {222}, series = {Proceedings of Machine Learning Research}, month = {11--14 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v222/zhang24a/zhang24a.pdf}, url = {https://proceedings.mlr.press/v222/zhang24a.html}, abstract = {Graph neural networks (GNNs) have demonstrated their great power in learning graph-structured data. Due to the limitations of expensive labeled data, contrastive learning has been applied in graph domain. We propose GWGCL, a graph contrastive learning method based on feature group whitening to achieve two key properties of contrastive learning: alignment and uniformity. GWGCL achieves the alignment by ensuring consistency between positive samples. There is no need for negative samples to participate, but rather to achieve the uniformity between samples through whitening. Because whitening has the effect of feature divergence, it avoids the collapse of all sample representations to a single point, which is called dimensional collapse. Moreover, GWGCL can achieve better results and higher efficiency without the need for asymmetric networks, projection layers, stopping gradients and complex loss function. Through extensive experiments, GWGCL performs competitively on node classification and graph classification tasks across ten common graph datasets.The code is in: https://github.com/MR9812/GWGCL.} }
Endnote
%0 Conference Paper %T Graph Contrastive Learning with Group Whitening %A Chunhui Zhang %A Rui Miao %B Proceedings of the 15th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Berrin Yanıkoğlu %E Wray Buntine %F pmlr-v222-zhang24a %I PMLR %P 1622--1637 %U https://proceedings.mlr.press/v222/zhang24a.html %V 222 %X Graph neural networks (GNNs) have demonstrated their great power in learning graph-structured data. Due to the limitations of expensive labeled data, contrastive learning has been applied in graph domain. We propose GWGCL, a graph contrastive learning method based on feature group whitening to achieve two key properties of contrastive learning: alignment and uniformity. GWGCL achieves the alignment by ensuring consistency between positive samples. There is no need for negative samples to participate, but rather to achieve the uniformity between samples through whitening. Because whitening has the effect of feature divergence, it avoids the collapse of all sample representations to a single point, which is called dimensional collapse. Moreover, GWGCL can achieve better results and higher efficiency without the need for asymmetric networks, projection layers, stopping gradients and complex loss function. Through extensive experiments, GWGCL performs competitively on node classification and graph classification tasks across ten common graph datasets.The code is in: https://github.com/MR9812/GWGCL.
APA
Zhang, C. & Miao, R.. (2024). Graph Contrastive Learning with Group Whitening. Proceedings of the 15th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 222:1622-1637 Available from https://proceedings.mlr.press/v222/zhang24a.html.

Related Material