Text2Edge: Language-Aware Temporal Graph Transformer for Dynamic Link Prediction

Dynamic link prediction in information networks (e.g., email, citation, social, and Wiki-pedia graphs) requires jointly modeling evolving topology and node-level semantics. However, incorporating language signals directly into temporal attention remains an open challenge. Many existing approaches either ignore textual information or attach static language features outside the attention mechanism, while naively using LLM-derived embeddings can be computationally costly and unstable without structural grounding. We introduce Text2Edge, a language-aware graph transformer that injects pretrained language representations into edge-sparse temporal attention, enabling semantic signals to influence how dynamic edges are weighted over time while preserving computational efficiency. Unlike purely language-based models or structure-only transformers, Text2Edge integrates semantic and structural information through a gated fusion mechanism, allowing the model to adaptively balance topology and language signals. To understand the role of semantics in dynamic link prediction, we conduct a controlled comparison between structural, semantic, and hybrid approaches. We evaluate Text2Edge alongside a strong structure-only transformer baseline (LPFormer) and language-augmented variants using BERT and LLaMa embeddings across four dynamic graph datasets. Our results show that structure-only models tend to plateau early, while semantic-aware models continue improving, indicating that semantic signals are critical in evolving real-world networks. The unified Text2Edge framework achieves the best overall performance, demonstrating that aligning pretrained language representations with edge-sparse temporal reasoning improves ranking quality and robustness without densifying the graph or fine-tuning the language encoder.