From Hawkes Processes to Attention: Time-Modulated Mechanisms for Event Sequences

Xinzi Tan, Kejian Zhang, Junhan Yu, Doudou Zhou
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:2584-2592, 2026.

Abstract

Marked Temporal Point Processes (MTPPs) arise naturally in medical, social, commercial, and financial domains. However, existing Transformer-based methods mostly inject temporal information only via positional encodings, relying on shared or parametric decay structures, which limits their ability to capture heterogeneous and type-specific temporal effects. Inspired by this observation, we derive a novel attention operator called Hawkes Attention from the multivariate Hawkes process theory for MTPP, using learnable per-type neural kernels to modulate query, key and value projections, thereby replacing the corresponding parts in the traditional attention. Benefited from the design, Hawkes Attention unifies event timing and content interaction, learning both the time-relevant behavior and type-specific excitation patterns from the data. The experimental results show that our method achieves better performance compared to the baselines. In addition to the general MTPP, our attention mechanism can also be easily applied to specific temporal structures, such as time series forecasting.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-tan26a, title = { From Hawkes Processes to Attention: Time-Modulated Mechanisms for Event Sequences }, author = {Tan, Xinzi and Zhang, Kejian and Yu, Junhan and Zhou, Doudou}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {2584--2592}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/tan26a/tan26a.pdf}, url = {https://proceedings.mlr.press/v300/tan26a.html}, abstract = { Marked Temporal Point Processes (MTPPs) arise naturally in medical, social, commercial, and financial domains. However, existing Transformer-based methods mostly inject temporal information only via positional encodings, relying on shared or parametric decay structures, which limits their ability to capture heterogeneous and type-specific temporal effects. Inspired by this observation, we derive a novel attention operator called Hawkes Attention from the multivariate Hawkes process theory for MTPP, using learnable per-type neural kernels to modulate query, key and value projections, thereby replacing the corresponding parts in the traditional attention. Benefited from the design, Hawkes Attention unifies event timing and content interaction, learning both the time-relevant behavior and type-specific excitation patterns from the data. The experimental results show that our method achieves better performance compared to the baselines. In addition to the general MTPP, our attention mechanism can also be easily applied to specific temporal structures, such as time series forecasting. } }
Endnote
%0 Conference Paper %T From Hawkes Processes to Attention: Time-Modulated Mechanisms for Event Sequences %A Xinzi Tan %A Kejian Zhang %A Junhan Yu %A Doudou Zhou %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-tan26a %I PMLR %P 2584--2592 %U https://proceedings.mlr.press/v300/tan26a.html %V 300 %X Marked Temporal Point Processes (MTPPs) arise naturally in medical, social, commercial, and financial domains. However, existing Transformer-based methods mostly inject temporal information only via positional encodings, relying on shared or parametric decay structures, which limits their ability to capture heterogeneous and type-specific temporal effects. Inspired by this observation, we derive a novel attention operator called Hawkes Attention from the multivariate Hawkes process theory for MTPP, using learnable per-type neural kernels to modulate query, key and value projections, thereby replacing the corresponding parts in the traditional attention. Benefited from the design, Hawkes Attention unifies event timing and content interaction, learning both the time-relevant behavior and type-specific excitation patterns from the data. The experimental results show that our method achieves better performance compared to the baselines. In addition to the general MTPP, our attention mechanism can also be easily applied to specific temporal structures, such as time series forecasting.
APA
Tan, X., Zhang, K., Yu, J. & Zhou, D.. (2026). From Hawkes Processes to Attention: Time-Modulated Mechanisms for Event Sequences . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:2584-2592 Available from https://proceedings.mlr.press/v300/tan26a.html.

Related Material