G-LaD: Graph-Language alignment for few-shot Diagnosis from fMRI

Abhishek Gupta, Vipul Kumar Singh, Jyotismita Barman, Sandeep Kumar, Anish Arora
Proceedings of the First Workshop on NeuroAI Multimodal Intelligence @ AAAI 2026, PMLR 308:180-188, 2026.

Abstract

Decoding human cognitive states from neural activity is a core challenge in artificial intelligence and computational neuroscience. Functional Magnetic Resonance Imaging (fMRI) captures high-dimensional spatiotemporal patterns of brain activity, yet characterizing cognitive states based on modeling the complex, dynamic dependencies among distributed regions remains difficult. While Graph Neural Networks (GNNs) to represent the brain as a structured graph has advanced functional connectivity (FC) analysis, they suffer from limited generalization, reliance on large labeled datasets, and poor transferability across neuro-imaging tasks. We introduce Graph-Language alignment for Diagnosis (G-LaD), that integrates graph representation learning with Large Language Models (LLMs) for data-efficient brain graph classification. G-LaD first pretrains a graph encoder, built upon Graph Isomorphism Network layers using a reconstruction-driven Denoising Autoencoder, to capture structural and topological invariants. In the second stage, distribution-level alignment between graph and language representations is achieved via a Sinkhorn-divergence objective, enabling smooth and transferable cross-modal mapping. Finally, a Chain-of-Thought prompting mechanism guides the LLM to perform reasoning-driven predictions. Empirical evaluations on the ABIDE dataset demonstrate superior few-shot generalization and robust performance of G-LAD in neuro-degenerative disorder classification.

Cite this Paper

BibTeX
@InProceedings{pmlr-v308-gupta26a, title = {G-LaD: Graph-Language alignment for few-shot Diagnosis from fMRI}, author = {Gupta, Abhishek and Singh, Vipul Kumar and Barman, Jyotismita and Kumar, Sandeep and Arora, Anish}, booktitle = {Proceedings of the First Workshop on NeuroAI Multimodal Intelligence @ AAAI 2026}, pages = {180--188}, year = {2026}, editor = {Abbasi-Asl, Reza and Iqbal, Asim and Ito, Shinya and Arkhipov, Anton and Sanborn, Sophia}, volume = {308}, series = {Proceedings of Machine Learning Research}, month = {27 Jan}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v308/main/assets/gupta26a/gupta26a.pdf}, url = {https://proceedings.mlr.press/v308/gupta26a.html}, abstract = {Decoding human cognitive states from neural activity is a core challenge in artificial intelligence and computational neuroscience. Functional Magnetic Resonance Imaging (fMRI) captures high-dimensional spatiotemporal patterns of brain activity, yet characterizing cognitive states based on modeling the complex, dynamic dependencies among distributed regions remains difficult. While Graph Neural Networks (GNNs) to represent the brain as a structured graph has advanced functional connectivity (FC) analysis, they suffer from limited generalization, reliance on large labeled datasets, and poor transferability across neuro-imaging tasks. We introduce Graph-Language alignment for Diagnosis (G-LaD), that integrates graph representation learning with Large Language Models (LLMs) for data-efficient brain graph classification. G-LaD first pretrains a graph encoder, built upon Graph Isomorphism Network layers using a reconstruction-driven Denoising Autoencoder, to capture structural and topological invariants. In the second stage, distribution-level alignment between graph and language representations is achieved via a Sinkhorn-divergence objective, enabling smooth and transferable cross-modal mapping. Finally, a Chain-of-Thought prompting mechanism guides the LLM to perform reasoning-driven predictions. Empirical evaluations on the ABIDE dataset demonstrate superior few-shot generalization and robust performance of G-LAD in neuro-degenerative disorder classification.} }
Endnote
%0 Conference Paper %T G-LaD: Graph-Language alignment for few-shot Diagnosis from fMRI %A Abhishek Gupta %A Vipul Kumar Singh %A Jyotismita Barman %A Sandeep Kumar %A Anish Arora %B Proceedings of the First Workshop on NeuroAI Multimodal Intelligence @ AAAI 2026 %C Proceedings of Machine Learning Research %D 2026 %E Reza Abbasi-Asl %E Asim Iqbal %E Shinya Ito %E Anton Arkhipov %E Sophia Sanborn %F pmlr-v308-gupta26a %I PMLR %P 180--188 %U https://proceedings.mlr.press/v308/gupta26a.html %V 308 %X Decoding human cognitive states from neural activity is a core challenge in artificial intelligence and computational neuroscience. Functional Magnetic Resonance Imaging (fMRI) captures high-dimensional spatiotemporal patterns of brain activity, yet characterizing cognitive states based on modeling the complex, dynamic dependencies among distributed regions remains difficult. While Graph Neural Networks (GNNs) to represent the brain as a structured graph has advanced functional connectivity (FC) analysis, they suffer from limited generalization, reliance on large labeled datasets, and poor transferability across neuro-imaging tasks. We introduce Graph-Language alignment for Diagnosis (G-LaD), that integrates graph representation learning with Large Language Models (LLMs) for data-efficient brain graph classification. G-LaD first pretrains a graph encoder, built upon Graph Isomorphism Network layers using a reconstruction-driven Denoising Autoencoder, to capture structural and topological invariants. In the second stage, distribution-level alignment between graph and language representations is achieved via a Sinkhorn-divergence objective, enabling smooth and transferable cross-modal mapping. Finally, a Chain-of-Thought prompting mechanism guides the LLM to perform reasoning-driven predictions. Empirical evaluations on the ABIDE dataset demonstrate superior few-shot generalization and robust performance of G-LAD in neuro-degenerative disorder classification.
APA
Gupta, A., Singh, V.K., Barman, J., Kumar, S. & Arora, A.. (2026). G-LaD: Graph-Language alignment for few-shot Diagnosis from fMRI. Proceedings of the First Workshop on NeuroAI Multimodal Intelligence @ AAAI 2026, in Proceedings of Machine Learning Research 308:180-188 Available from https://proceedings.mlr.press/v308/gupta26a.html.

Related Material