SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process

Zichong Li, Yanbo Xu, Simiao Zuo, Haoming Jiang, Chao Zhang, Tuo Zhao, Hongyuan Zha
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:20210-20220, 2023.

Abstract

Transformer Hawkes process models have shown to be successful in modeling event sequence data. However, most of the existing training methods rely on maximizing the likelihood of event sequences, which involves calculating some intractable integral. Moreover, the existing methods fail to provide uncertainty quantification for model predictions, e.g., confidence interval for the predicted event’s arrival time. To address these issues, we propose SMURF-THP, a score-based method for learning Transformer Hawkes process and quantifying prediction uncertainty. Specifically, SMURF-THP learns the score function of the event’s arrival time based on a score-matching objective that avoids the intractable computation. With such a learnt score function, we can sample arrival time of events from the predictive distribution. This naturally allows for the quantification of uncertainty by computing confidence intervals over the generated samples. We conduct extensive experiments in both event type prediction and uncertainty quantification on time of arrival. In all the experiments, SMURF-THP outperforms existing likelihood-based methods in confidence calibration while exhibiting comparable prediction accuracy.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-li23aj, title = {{SMURF}-{THP}: Score Matching-based {U}nce{R}tainty quanti{F}ication for Transformer {H}awkes Process}, author = {Li, Zichong and Xu, Yanbo and Zuo, Simiao and Jiang, Haoming and Zhang, Chao and Zhao, Tuo and Zha, Hongyuan}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {20210--20220}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/li23aj/li23aj.pdf}, url = {https://proceedings.mlr.press/v202/li23aj.html}, abstract = {Transformer Hawkes process models have shown to be successful in modeling event sequence data. However, most of the existing training methods rely on maximizing the likelihood of event sequences, which involves calculating some intractable integral. Moreover, the existing methods fail to provide uncertainty quantification for model predictions, e.g., confidence interval for the predicted event’s arrival time. To address these issues, we propose SMURF-THP, a score-based method for learning Transformer Hawkes process and quantifying prediction uncertainty. Specifically, SMURF-THP learns the score function of the event’s arrival time based on a score-matching objective that avoids the intractable computation. With such a learnt score function, we can sample arrival time of events from the predictive distribution. This naturally allows for the quantification of uncertainty by computing confidence intervals over the generated samples. We conduct extensive experiments in both event type prediction and uncertainty quantification on time of arrival. In all the experiments, SMURF-THP outperforms existing likelihood-based methods in confidence calibration while exhibiting comparable prediction accuracy.} }
Endnote
%0 Conference Paper %T SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process %A Zichong Li %A Yanbo Xu %A Simiao Zuo %A Haoming Jiang %A Chao Zhang %A Tuo Zhao %A Hongyuan Zha %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-li23aj %I PMLR %P 20210--20220 %U https://proceedings.mlr.press/v202/li23aj.html %V 202 %X Transformer Hawkes process models have shown to be successful in modeling event sequence data. However, most of the existing training methods rely on maximizing the likelihood of event sequences, which involves calculating some intractable integral. Moreover, the existing methods fail to provide uncertainty quantification for model predictions, e.g., confidence interval for the predicted event’s arrival time. To address these issues, we propose SMURF-THP, a score-based method for learning Transformer Hawkes process and quantifying prediction uncertainty. Specifically, SMURF-THP learns the score function of the event’s arrival time based on a score-matching objective that avoids the intractable computation. With such a learnt score function, we can sample arrival time of events from the predictive distribution. This naturally allows for the quantification of uncertainty by computing confidence intervals over the generated samples. We conduct extensive experiments in both event type prediction and uncertainty quantification on time of arrival. In all the experiments, SMURF-THP outperforms existing likelihood-based methods in confidence calibration while exhibiting comparable prediction accuracy.
APA
Li, Z., Xu, Y., Zuo, S., Jiang, H., Zhang, C., Zhao, T. & Zha, H.. (2023). SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:20210-20220 Available from https://proceedings.mlr.press/v202/li23aj.html.

Related Material