Representation Learning on Biomolecular Structures Using Equivariant Graph Attention

Tuan Le, Frank Noe, Djork-Arné Clevert
Proceedings of the First Learning on Graphs Conference, PMLR 198:30:1-30:17, 2022.

Abstract

Learning and reasoning about 3D molecular structures with varying size is an emerging and important challenge in machine learning and especially in the development of biotherapeutics. Equivariant Graph Neural Networks (GNNs) can simultaneously leverage the geometric and relational detail of the problem domain and are known to learn expressive representations through the propagation of information between nodes leveraging higher-order representations to faithfully express the geometry of the data, such as directionality in their intermediate layers. In this work, we propose an equivariant GNN that operates with Cartesian coordinates to incorporate directionality and we implement a novel attention mechanism, acting as a content and spatial dependent filter when propagating information between nodes. Our proposed message function processes vector features in a geometrically meaningful way by mixing existing vectors and creating new ones based on cross products. We demonstrate the efficacy of our architecture on accurately predicting properties of large biomolecules and show its computational advantage over recent methods which rely on irreducible representations by means of the spherical harmonics expansion.

Cite this Paper

BibTeX
@InProceedings{pmlr-v198-le22a, title = {Representation Learning on Biomolecular Structures Using Equivariant Graph Attention}, author = {Le, Tuan and Noe, Frank and Clevert, Djork-Arn{\'e}}, booktitle = {Proceedings of the First Learning on Graphs Conference}, pages = {30:1--30:17}, year = {2022}, editor = {Rieck, Bastian and Pascanu, Razvan}, volume = {198}, series = {Proceedings of Machine Learning Research}, month = {09--12 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v198/le22a/le22a.pdf}, url = {https://proceedings.mlr.press/v198/le22a.html}, abstract = {Learning and reasoning about 3D molecular structures with varying size is an emerging and important challenge in machine learning and especially in the development of biotherapeutics. Equivariant Graph Neural Networks (GNNs) can simultaneously leverage the geometric and relational detail of the problem domain and are known to learn expressive representations through the propagation of information between nodes leveraging higher-order representations to faithfully express the geometry of the data, such as directionality in their intermediate layers. In this work, we propose an equivariant GNN that operates with Cartesian coordinates to incorporate directionality and we implement a novel attention mechanism, acting as a content and spatial dependent filter when propagating information between nodes. Our proposed message function processes vector features in a geometrically meaningful way by mixing existing vectors and creating new ones based on cross products. We demonstrate the efficacy of our architecture on accurately predicting properties of large biomolecules and show its computational advantage over recent methods which rely on irreducible representations by means of the spherical harmonics expansion.} }
Endnote
%0 Conference Paper %T Representation Learning on Biomolecular Structures Using Equivariant Graph Attention %A Tuan Le %A Frank Noe %A Djork-Arné Clevert %B Proceedings of the First Learning on Graphs Conference %C Proceedings of Machine Learning Research %D 2022 %E Bastian Rieck %E Razvan Pascanu %F pmlr-v198-le22a %I PMLR %P 30:1--30:17 %U https://proceedings.mlr.press/v198/le22a.html %V 198 %X Learning and reasoning about 3D molecular structures with varying size is an emerging and important challenge in machine learning and especially in the development of biotherapeutics. Equivariant Graph Neural Networks (GNNs) can simultaneously leverage the geometric and relational detail of the problem domain and are known to learn expressive representations through the propagation of information between nodes leveraging higher-order representations to faithfully express the geometry of the data, such as directionality in their intermediate layers. In this work, we propose an equivariant GNN that operates with Cartesian coordinates to incorporate directionality and we implement a novel attention mechanism, acting as a content and spatial dependent filter when propagating information between nodes. Our proposed message function processes vector features in a geometrically meaningful way by mixing existing vectors and creating new ones based on cross products. We demonstrate the efficacy of our architecture on accurately predicting properties of large biomolecules and show its computational advantage over recent methods which rely on irreducible representations by means of the spherical harmonics expansion.
APA
Le, T., Noe, F. & Clevert, D.. (2022). Representation Learning on Biomolecular Structures Using Equivariant Graph Attention. Proceedings of the First Learning on Graphs Conference, in Proceedings of Machine Learning Research 198:30:1-30:17 Available from https://proceedings.mlr.press/v198/le22a.html.

Related Material