Deep Fourier Kernel for Self-Attentive Point Processes

Shixiang Zhu, Minghe Zhang, Ruyi Ding, Yao Xie
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:856-864, 2021.

Abstract

We present a novel attention-based model for discrete event data to capture complex non-linear temporal dependence structures. We borrow the idea from the attention mechanism and incorporate it into the point processes’ conditional intensity function. We further introduce a novel score function using Fourier kernel embedding, whose spectrum is represented using neural networks, which drastically differs from the traditional dot-product kernel and can capture a more complex similarity structure. We establish our approach’s theoretical properties and demonstrate our approach’s competitive performance compared to the state-of-the-art for synthetic and real data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-zhu21b, title = { Deep Fourier Kernel for Self-Attentive Point Processes }, author = {Zhu, Shixiang and Zhang, Minghe and Ding, Ruyi and Xie, Yao}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {856--864}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/zhu21b/zhu21b.pdf}, url = {https://proceedings.mlr.press/v130/zhu21b.html}, abstract = { We present a novel attention-based model for discrete event data to capture complex non-linear temporal dependence structures. We borrow the idea from the attention mechanism and incorporate it into the point processes’ conditional intensity function. We further introduce a novel score function using Fourier kernel embedding, whose spectrum is represented using neural networks, which drastically differs from the traditional dot-product kernel and can capture a more complex similarity structure. We establish our approach’s theoretical properties and demonstrate our approach’s competitive performance compared to the state-of-the-art for synthetic and real data. } }
Endnote
%0 Conference Paper %T Deep Fourier Kernel for Self-Attentive Point Processes %A Shixiang Zhu %A Minghe Zhang %A Ruyi Ding %A Yao Xie %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-zhu21b %I PMLR %P 856--864 %U https://proceedings.mlr.press/v130/zhu21b.html %V 130 %X We present a novel attention-based model for discrete event data to capture complex non-linear temporal dependence structures. We borrow the idea from the attention mechanism and incorporate it into the point processes’ conditional intensity function. We further introduce a novel score function using Fourier kernel embedding, whose spectrum is represented using neural networks, which drastically differs from the traditional dot-product kernel and can capture a more complex similarity structure. We establish our approach’s theoretical properties and demonstrate our approach’s competitive performance compared to the state-of-the-art for synthetic and real data.
APA
Zhu, S., Zhang, M., Ding, R. & Xie, Y.. (2021). Deep Fourier Kernel for Self-Attentive Point Processes . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:856-864 Available from https://proceedings.mlr.press/v130/zhu21b.html.

Related Material