Weisfeiler Leman for Euclidean Equivariant Machine Learning

Snir Hordan, Tal Amir, Nadav Dym
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:18749-18784, 2024.

Abstract

The k-Weisfeiler-Leman (k-WL) graph isomorphism test hierarchy is a common method for assessing the expressive power of graph neural networks (GNNs). Recently, GNNs whose expressive power is equivalent to the 2-WL test were proven to be universal on weighted graphs which encode 3D point cloud data, yet this result is limited to invariant continuous functions on point clouds. In this paper, we extend this result in three ways: Firstly, we show that PPGN can simulate 2-WL uniformly on all point clouds with low complexity. Secondly, we show that 2-WL tests can be extended to point clouds which include both positions and velocities, a scenario often encountered in applications. Finally, we provide a general framework for proving equivariant universality and leverage it to prove that a simple modification of this invariant PPGN architecture can be used to obtain a universal equivariant architecture that can approximate all continuous equivariant functions uniformly. Building on our results, we develop our WeLNet architecture, which sets new state-of-the-art results on the N-Body dynamics task and the GEOM-QM9 molecular conformation generation task.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-hordan24a, title = {Weisfeiler Leman for {E}uclidean Equivariant Machine Learning}, author = {Hordan, Snir and Amir, Tal and Dym, Nadav}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {18749--18784}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/hordan24a/hordan24a.pdf}, url = {https://proceedings.mlr.press/v235/hordan24a.html}, abstract = {The $k$-Weisfeiler-Leman ($k$-WL) graph isomorphism test hierarchy is a common method for assessing the expressive power of graph neural networks (GNNs). Recently, GNNs whose expressive power is equivalent to the $2$-WL test were proven to be universal on weighted graphs which encode $3\mathrm{D}$ point cloud data, yet this result is limited to invariant continuous functions on point clouds. In this paper, we extend this result in three ways: Firstly, we show that PPGN can simulate $2$-WL uniformly on all point clouds with low complexity. Secondly, we show that $2$-WL tests can be extended to point clouds which include both positions and velocities, a scenario often encountered in applications. Finally, we provide a general framework for proving equivariant universality and leverage it to prove that a simple modification of this invariant PPGN architecture can be used to obtain a universal equivariant architecture that can approximate all continuous equivariant functions uniformly. Building on our results, we develop our WeLNet architecture, which sets new state-of-the-art results on the N-Body dynamics task and the GEOM-QM9 molecular conformation generation task.} }
Endnote
%0 Conference Paper %T Weisfeiler Leman for Euclidean Equivariant Machine Learning %A Snir Hordan %A Tal Amir %A Nadav Dym %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-hordan24a %I PMLR %P 18749--18784 %U https://proceedings.mlr.press/v235/hordan24a.html %V 235 %X The $k$-Weisfeiler-Leman ($k$-WL) graph isomorphism test hierarchy is a common method for assessing the expressive power of graph neural networks (GNNs). Recently, GNNs whose expressive power is equivalent to the $2$-WL test were proven to be universal on weighted graphs which encode $3\mathrm{D}$ point cloud data, yet this result is limited to invariant continuous functions on point clouds. In this paper, we extend this result in three ways: Firstly, we show that PPGN can simulate $2$-WL uniformly on all point clouds with low complexity. Secondly, we show that $2$-WL tests can be extended to point clouds which include both positions and velocities, a scenario often encountered in applications. Finally, we provide a general framework for proving equivariant universality and leverage it to prove that a simple modification of this invariant PPGN architecture can be used to obtain a universal equivariant architecture that can approximate all continuous equivariant functions uniformly. Building on our results, we develop our WeLNet architecture, which sets new state-of-the-art results on the N-Body dynamics task and the GEOM-QM9 molecular conformation generation task.
APA
Hordan, S., Amir, T. & Dym, N.. (2024). Weisfeiler Leman for Euclidean Equivariant Machine Learning. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:18749-18784 Available from https://proceedings.mlr.press/v235/hordan24a.html.

Related Material