Transformer-Based Deep Survival Analysis

Shi Hu, Egill Fridgeirsson, Guido van Wingen, Max Welling
Proceedings of AAAI Spring Symposium on Survival Prediction - Algorithms, Challenges, and Applications 2021, PMLR 146:132-148, 2021.

Abstract

In this work, we propose a new Transformer-based survival model which estimates the patient-specific survival distribution. Our contributions are twofold. First, to the best of our knowledge, existing deep survival models use either fully connected or recurrent networks, and we are the first to apply the Transformer in survival analysis. In addition, we use ordinal regression to optimize the survival probabilities over time, and penalize randomized discordant pairs. Second, many survival models are evaluated using only the ranking metrics such as the concordance index. We propose to also use the absolute error metric that evaluates the precise duration predictions on observed subjects. We demonstrate our model on two publicly available real-world datasets, and show that our mean absolute error results are significantly better than the current models, meanwhile, it is challenging to determine the best model under the concordance index.

Cite this Paper

BibTeX
@InProceedings{pmlr-v146-hu21a, title = {Transformer-Based Deep Survival Analysis}, author = {Hu, Shi and Fridgeirsson, Egill and Wingen, Guido van and Welling, Max}, booktitle = {Proceedings of AAAI Spring Symposium on Survival Prediction - Algorithms, Challenges, and Applications 2021}, pages = {132--148}, year = {2021}, editor = {Greiner, Russell and Kumar, Neeraj and Gerds, Thomas Alexander and van der Schaar, Mihaela}, volume = {146}, series = {Proceedings of Machine Learning Research}, month = {22--24 Mar}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v146/hu21a/hu21a.pdf}, url = {https://proceedings.mlr.press/v146/hu21a.html}, abstract = {In this work, we propose a new Transformer-based survival model which estimates the patient-specific survival distribution. Our contributions are twofold. First, to the best of our knowledge, existing deep survival models use either fully connected or recurrent networks, and we are the first to apply the Transformer in survival analysis. In addition, we use ordinal regression to optimize the survival probabilities over time, and penalize randomized discordant pairs. Second, many survival models are evaluated using only the ranking metrics such as the concordance index. We propose to also use the absolute error metric that evaluates the precise duration predictions on observed subjects. We demonstrate our model on two publicly available real-world datasets, and show that our mean absolute error results are significantly better than the current models, meanwhile, it is challenging to determine the best model under the concordance index.} }
Endnote
%0 Conference Paper %T Transformer-Based Deep Survival Analysis %A Shi Hu %A Egill Fridgeirsson %A Guido van Wingen %A Max Welling %B Proceedings of AAAI Spring Symposium on Survival Prediction - Algorithms, Challenges, and Applications 2021 %C Proceedings of Machine Learning Research %D 2021 %E Russell Greiner %E Neeraj Kumar %E Thomas Alexander Gerds %E Mihaela van der Schaar %F pmlr-v146-hu21a %I PMLR %P 132--148 %U https://proceedings.mlr.press/v146/hu21a.html %V 146 %X In this work, we propose a new Transformer-based survival model which estimates the patient-specific survival distribution. Our contributions are twofold. First, to the best of our knowledge, existing deep survival models use either fully connected or recurrent networks, and we are the first to apply the Transformer in survival analysis. In addition, we use ordinal regression to optimize the survival probabilities over time, and penalize randomized discordant pairs. Second, many survival models are evaluated using only the ranking metrics such as the concordance index. We propose to also use the absolute error metric that evaluates the precise duration predictions on observed subjects. We demonstrate our model on two publicly available real-world datasets, and show that our mean absolute error results are significantly better than the current models, meanwhile, it is challenging to determine the best model under the concordance index.
APA
Hu, S., Fridgeirsson, E., Wingen, G.v. & Welling, M.. (2021). Transformer-Based Deep Survival Analysis. Proceedings of AAAI Spring Symposium on Survival Prediction - Algorithms, Challenges, and Applications 2021, in Proceedings of Machine Learning Research 146:132-148 Available from https://proceedings.mlr.press/v146/hu21a.html.

Related Material