Stable Prediction on Graphs with Agnostic Distribution Shifts

Shengyu Zhang, Yunze Tong, Kun Kuang, Fuli Feng, Jiezhong Qiu, Jin Yu, Zhou Zhao, Hongxia Yang, Zhongfei Zhang, Fei Wu
Proceedings of The KDD'23 Workshop on Causal Discovery, Prediction and Decision, PMLR 218:49-74, 2023.

Abstract

Most graph neural networks (GNNs) are proposed and evaluated under independent and identically distributed (IID) training and testing data. In real-world applications, however, agnostic distribution shifts from training to testing naturally exist, leading to unstable prediction of traditional GNNs. To bridge the gap, we pursue stable prediction on graphs, i.e., to achieve high average performance and low performance variance (stability) across non-IID testing graphs. The key to stable prediction lies in capturing stable properties that are resilient to distribution shifts. In this light, we aim to identify neighbor nodes (properties) in neighborhood aggregation that are consistently important for prediction under heterogeneous distribution shifts. To achieve this target, we propose a model-agnostic stable learning framework for GNNs. The framework performs biased selection on the observed training graph, resulting in multiple non-IID graph subsets. We train one weight predictor per subset to measure the importance of properties under a particular distribution shift, and multiple predictors could tell the properties that are consistently important. An important property should contribute to high average performance and also stability (low performance variance) across non-IID subsets. In this regard, in training importance predictors, we introduce a globally stable regularizer to reduce the variance of training losses across non-IID graph datasets. Based on the importance weights of properties across non- IID subsets, a locally stable regularizer down-weights unstable properties in prediction. We conduct extensive experiments on several graph benchmarks and a noisy industrial recommendation dataset where distribution shifts exist. The results demonstrate that our method outperforms various state-of-the-art GNNs for stable prediction on graphs with agnostic distribution shifts.

Cite this Paper

BibTeX
@InProceedings{pmlr-v218-zhang23a, title = {Stable Prediction on Graphs with Agnostic Distribution Shifts}, author = {Zhang, Shengyu and Tong, Yunze and Kuang, Kun and Feng, Fuli and Qiu, Jiezhong and Yu, Jin and Zhao, Zhou and Yang, Hongxia and Zhang, Zhongfei and Wu, Fei}, booktitle = {Proceedings of The KDD'23 Workshop on Causal Discovery, Prediction and Decision}, pages = {49--74}, year = {2023}, editor = {Le, Thuc and Li, Jiuyong and Ness, Robert and Triantafillou, Sofia and Shimizu, Shohei and Cui, Peng and Kuang, Kun and Pei, Jian and Wang, Fei and Prosperi, Mattia}, volume = {218}, series = {Proceedings of Machine Learning Research}, month = {07 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v218/zhang23a/zhang23a.pdf}, url = {https://proceedings.mlr.press/v218/zhang23a.html}, abstract = {Most graph neural networks (GNNs) are proposed and evaluated under independent and identically distributed (IID) training and testing data. In real-world applications, however, agnostic distribution shifts from training to testing naturally exist, leading to unstable prediction of traditional GNNs. To bridge the gap, we pursue stable prediction on graphs, i.e., to achieve high average performance and low performance variance (stability) across non-IID testing graphs. The key to stable prediction lies in capturing stable properties that are resilient to distribution shifts. In this light, we aim to identify neighbor nodes (properties) in neighborhood aggregation that are consistently important for prediction under heterogeneous distribution shifts. To achieve this target, we propose a model-agnostic stable learning framework for GNNs. The framework performs biased selection on the observed training graph, resulting in multiple non-IID graph subsets. We train one weight predictor per subset to measure the importance of properties under a particular distribution shift, and multiple predictors could tell the properties that are consistently important. An important property should contribute to high average performance and also stability (low performance variance) across non-IID subsets. In this regard, in training importance predictors, we introduce a globally stable regularizer to reduce the variance of training losses across non-IID graph datasets. Based on the importance weights of properties across non- IID subsets, a locally stable regularizer down-weights unstable properties in prediction. We conduct extensive experiments on several graph benchmarks and a noisy industrial recommendation dataset where distribution shifts exist. The results demonstrate that our method outperforms various state-of-the-art GNNs for stable prediction on graphs with agnostic distribution shifts.} }
Endnote
%0 Conference Paper %T Stable Prediction on Graphs with Agnostic Distribution Shifts %A Shengyu Zhang %A Yunze Tong %A Kun Kuang %A Fuli Feng %A Jiezhong Qiu %A Jin Yu %A Zhou Zhao %A Hongxia Yang %A Zhongfei Zhang %A Fei Wu %B Proceedings of The KDD'23 Workshop on Causal Discovery, Prediction and Decision %C Proceedings of Machine Learning Research %D 2023 %E Thuc Le %E Jiuyong Li %E Robert Ness %E Sofia Triantafillou %E Shohei Shimizu %E Peng Cui %E Kun Kuang %E Jian Pei %E Fei Wang %E Mattia Prosperi %F pmlr-v218-zhang23a %I PMLR %P 49--74 %U https://proceedings.mlr.press/v218/zhang23a.html %V 218 %X Most graph neural networks (GNNs) are proposed and evaluated under independent and identically distributed (IID) training and testing data. In real-world applications, however, agnostic distribution shifts from training to testing naturally exist, leading to unstable prediction of traditional GNNs. To bridge the gap, we pursue stable prediction on graphs, i.e., to achieve high average performance and low performance variance (stability) across non-IID testing graphs. The key to stable prediction lies in capturing stable properties that are resilient to distribution shifts. In this light, we aim to identify neighbor nodes (properties) in neighborhood aggregation that are consistently important for prediction under heterogeneous distribution shifts. To achieve this target, we propose a model-agnostic stable learning framework for GNNs. The framework performs biased selection on the observed training graph, resulting in multiple non-IID graph subsets. We train one weight predictor per subset to measure the importance of properties under a particular distribution shift, and multiple predictors could tell the properties that are consistently important. An important property should contribute to high average performance and also stability (low performance variance) across non-IID subsets. In this regard, in training importance predictors, we introduce a globally stable regularizer to reduce the variance of training losses across non-IID graph datasets. Based on the importance weights of properties across non- IID subsets, a locally stable regularizer down-weights unstable properties in prediction. We conduct extensive experiments on several graph benchmarks and a noisy industrial recommendation dataset where distribution shifts exist. The results demonstrate that our method outperforms various state-of-the-art GNNs for stable prediction on graphs with agnostic distribution shifts.
APA
Zhang, S., Tong, Y., Kuang, K., Feng, F., Qiu, J., Yu, J., Zhao, Z., Yang, H., Zhang, Z. & Wu, F.. (2023). Stable Prediction on Graphs with Agnostic Distribution Shifts. Proceedings of The KDD'23 Workshop on Causal Discovery, Prediction and Decision, in Proceedings of Machine Learning Research 218:49-74 Available from https://proceedings.mlr.press/v218/zhang23a.html.

Related Material