Learning Graph Node Embeddings by Smooth Pair Sampling

Konstantin Kutzkov
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:1828-1836, 2025.

Abstract

Random walk based node embedding algorithms have attracted a lot of attention due to their scalability and ease of implementation. Previous research has focused on different walk strategies, optimization objectives, and embedding learning models. Inspired by observations on real data, we take a different approach and propose a new regularization technique. More precisely, the frequencies of node pairs generated by the skip-gram model on random walk node sequences follow a highly skewed distribution which causes learning to be dominated by a fraction of the pairs. We address the issue by designing an efficient sampling procedure that generates node pairs according to their smoothed frequency. Theoretical and experimental results demonstrate the advantages of our approach.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-kutzkov25a, title = {Learning Graph Node Embeddings by Smooth Pair Sampling}, author = {Kutzkov, Konstantin}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {1828--1836}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/kutzkov25a/kutzkov25a.pdf}, url = {https://proceedings.mlr.press/v258/kutzkov25a.html}, abstract = {Random walk based node embedding algorithms have attracted a lot of attention due to their scalability and ease of implementation. Previous research has focused on different walk strategies, optimization objectives, and embedding learning models. Inspired by observations on real data, we take a different approach and propose a new regularization technique. More precisely, the frequencies of node pairs generated by the skip-gram model on random walk node sequences follow a highly skewed distribution which causes learning to be dominated by a fraction of the pairs. We address the issue by designing an efficient sampling procedure that generates node pairs according to their smoothed frequency. Theoretical and experimental results demonstrate the advantages of our approach.} }
Endnote
%0 Conference Paper %T Learning Graph Node Embeddings by Smooth Pair Sampling %A Konstantin Kutzkov %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-kutzkov25a %I PMLR %P 1828--1836 %U https://proceedings.mlr.press/v258/kutzkov25a.html %V 258 %X Random walk based node embedding algorithms have attracted a lot of attention due to their scalability and ease of implementation. Previous research has focused on different walk strategies, optimization objectives, and embedding learning models. Inspired by observations on real data, we take a different approach and propose a new regularization technique. More precisely, the frequencies of node pairs generated by the skip-gram model on random walk node sequences follow a highly skewed distribution which causes learning to be dominated by a fraction of the pairs. We address the issue by designing an efficient sampling procedure that generates node pairs according to their smoothed frequency. Theoretical and experimental results demonstrate the advantages of our approach.
APA
Kutzkov, K.. (2025). Learning Graph Node Embeddings by Smooth Pair Sampling. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:1828-1836 Available from https://proceedings.mlr.press/v258/kutzkov25a.html.

Related Material