Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration

Shi-Ang Qi, Yakun Yu, Russell Greiner
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:41303-41339, 2024.

Abstract

Discrimination and calibration represent two important properties of survival analysis, with the former assessing the model’s ability to accurately rank subjects and the latter evaluating the alignment of predicted outcomes with actual events. With their distinct nature, it is hard for survival models to simultaneously optimize both of them especially as many previous results found improving calibration tends to diminish discrimination performance. This paper introduces a novel approach utilizing conformal regression that can improve a model’s calibration without degrading discrimination. We provide theoretical guarantees for the above claim, and rigorously validate the efficiency of our approach across 11 real-world datasets, showcasing its practical applicability and robustness in diverse scenarios.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-qi24a, title = {Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration}, author = {Qi, Shi-Ang and Yu, Yakun and Greiner, Russell}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {41303--41339}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/qi24a/qi24a.pdf}, url = {https://proceedings.mlr.press/v235/qi24a.html}, abstract = {Discrimination and calibration represent two important properties of survival analysis, with the former assessing the model’s ability to accurately rank subjects and the latter evaluating the alignment of predicted outcomes with actual events. With their distinct nature, it is hard for survival models to simultaneously optimize both of them especially as many previous results found improving calibration tends to diminish discrimination performance. This paper introduces a novel approach utilizing conformal regression that can improve a model’s calibration without degrading discrimination. We provide theoretical guarantees for the above claim, and rigorously validate the efficiency of our approach across 11 real-world datasets, showcasing its practical applicability and robustness in diverse scenarios.} }
Endnote
%0 Conference Paper %T Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration %A Shi-Ang Qi %A Yakun Yu %A Russell Greiner %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-qi24a %I PMLR %P 41303--41339 %U https://proceedings.mlr.press/v235/qi24a.html %V 235 %X Discrimination and calibration represent two important properties of survival analysis, with the former assessing the model’s ability to accurately rank subjects and the latter evaluating the alignment of predicted outcomes with actual events. With their distinct nature, it is hard for survival models to simultaneously optimize both of them especially as many previous results found improving calibration tends to diminish discrimination performance. This paper introduces a novel approach utilizing conformal regression that can improve a model’s calibration without degrading discrimination. We provide theoretical guarantees for the above claim, and rigorously validate the efficiency of our approach across 11 real-world datasets, showcasing its practical applicability and robustness in diverse scenarios.
APA
Qi, S., Yu, Y. & Greiner, R.. (2024). Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:41303-41339 Available from https://proceedings.mlr.press/v235/qi24a.html.

Related Material