Fisher-Bures Adversary Graph Convolutional Networks

Ke Sun, Piotr Koniusz, Zhen Wang
Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, PMLR 115:465-475, 2020.

Abstract

In a graph convolutional network, we assume that the graph G is generated wrt some observation noise. During learning, we make small random perturbations ΔG of the graph and try to improve generalization. Based on quantum information geometry, ΔG can be characterized by the eigendecomposition of the graph Laplacian matrix. We try to minimize the loss wrt the perturbed G+ΔG while making ΔG to be effective in terms of the Fisher information of the neural network. Our proposed model can consistently improve graph convolutional networks on semi-supervised node classification tasks with reasonable computational overhead. We present three different geometries on the manifold of graphs: the intrinsic geometry measures the information theoretic dynamics of a graph; the extrinsic geometry characterizes how such dynamics can affect externally a graph neural network; the embedding geometry is for measuring node embeddings. These new analytical tools are useful in developing a good understanding of graph neural networks and fostering new techniques.

Cite this Paper

BibTeX
@InProceedings{pmlr-v115-sun20a, title = {Fisher-Bures Adversary Graph Convolutional Networks}, author = {Sun, Ke and Koniusz, Piotr and Wang, Zhen}, booktitle = {Proceedings of The 35th Uncertainty in Artificial Intelligence Conference}, pages = {465--475}, year = {2020}, editor = {Adams, Ryan P. and Gogate, Vibhav}, volume = {115}, series = {Proceedings of Machine Learning Research}, month = {22--25 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v115/sun20a/sun20a.pdf}, url = {https://proceedings.mlr.press/v115/sun20a.html}, abstract = {In a graph convolutional network, we assume that the graph $G$ is generated wrt some observation noise. During learning, we make small random perturbations $\Delta{}G$ of the graph and try to improve generalization. Based on quantum information geometry, $\Delta{}G$ can be characterized by the eigendecomposition of the graph Laplacian matrix. We try to minimize the loss wrt the perturbed $G+\Delta{G}$ while making $\Delta{G}$ to be effective in terms of the Fisher information of the neural network. Our proposed model can consistently improve graph convolutional networks on semi-supervised node classification tasks with reasonable computational overhead. We present three different geometries on the manifold of graphs: the intrinsic geometry measures the information theoretic dynamics of a graph; the extrinsic geometry characterizes how such dynamics can affect externally a graph neural network; the embedding geometry is for measuring node embeddings. These new analytical tools are useful in developing a good understanding of graph neural networks and fostering new techniques.} }
Endnote
%0 Conference Paper %T Fisher-Bures Adversary Graph Convolutional Networks %A Ke Sun %A Piotr Koniusz %A Zhen Wang %B Proceedings of The 35th Uncertainty in Artificial Intelligence Conference %C Proceedings of Machine Learning Research %D 2020 %E Ryan P. Adams %E Vibhav Gogate %F pmlr-v115-sun20a %I PMLR %P 465--475 %U https://proceedings.mlr.press/v115/sun20a.html %V 115 %X In a graph convolutional network, we assume that the graph $G$ is generated wrt some observation noise. During learning, we make small random perturbations $\Delta{}G$ of the graph and try to improve generalization. Based on quantum information geometry, $\Delta{}G$ can be characterized by the eigendecomposition of the graph Laplacian matrix. We try to minimize the loss wrt the perturbed $G+\Delta{G}$ while making $\Delta{G}$ to be effective in terms of the Fisher information of the neural network. Our proposed model can consistently improve graph convolutional networks on semi-supervised node classification tasks with reasonable computational overhead. We present three different geometries on the manifold of graphs: the intrinsic geometry measures the information theoretic dynamics of a graph; the extrinsic geometry characterizes how such dynamics can affect externally a graph neural network; the embedding geometry is for measuring node embeddings. These new analytical tools are useful in developing a good understanding of graph neural networks and fostering new techniques.
APA
Sun, K., Koniusz, P. & Wang, Z.. (2020). Fisher-Bures Adversary Graph Convolutional Networks. Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, in Proceedings of Machine Learning Research 115:465-475 Available from https://proceedings.mlr.press/v115/sun20a.html.

Related Material