Attention-to-Survival: Multimodal Fracture Risk Prediction Based on Pelvic Radiographs and Clinical Data from the Study of Osteoporotic Fractures

Niklas C. Koser, Marten J. Finck, Silja Janßen, Coenraad Mouton, Li-Y. Lui, Steven R. Cummings, Kevin Köser, Jan-B. Hövener, Sören Pirk, Claus-C. Glüer
Proceedings of The 9th International Conference on Medical Imaging with Deep Learning, PMLR 315:2640-2665, 2026.

Abstract

Osteoporotic changes in the hip structure render the proximal femur particularly vulnerable to fractures, which leads to severe consequences for patients’ health and significant socioeconomic burdens, a strongly increasing problem in aging populations. Accurate risk estimation is therefore essential for initiating timely preventive measures. However, the current clinical standard measures bone mineral density (BMD) and the Fracture Risk Assessment Tool (FRAX) provide only limited predictive value. Neither BMD nor FRAX capture structural characteristics that could be derived from pelvic radiographs that are widely available. To address this gap, we present the Attention-to-Survival Fusion (ATSF) model, a multimodal survival analysis framework that combines clinical risk factors (CRFs) with pelvic radiograph features. An attention-based architecture equipped with a deep conditional transformation model (DCTM) prediction head enables accurate estimation of time-dependent fracture risk. The ATSF model is designed to accommodate missing clinical variables, handle all forms of non-informative censoring, and provides modality-specific interpretability through the attention mechanisms. It was developed, validated and tested with data of 7825 women from the Study of Osteoporotic Fractures (SOF) followed for fracture incidence for 23 years. We benchmark ATSF against established baselines, including FRAX, the Cox proportional hazards model (CoxPH), and a deep learning reference model. Our results demonstrate significant superior performance across concordance index (C-index) and area under the receiver operating characteristic curve (AUC), indicating the importance of integrating radiographic and clinical data within a unified survival framework. Furthermore, offering improved interpretability and a scalable multimodal design, the proposed method provides a promising alternative for advancing individualized hip-fracture risk prediction in osteoporosis research and precision medicine.

Cite this Paper

BibTeX
@InProceedings{pmlr-v315-koser26a, title = {Attention-to-Survival: Multimodal Fracture Risk Prediction Based on Pelvic Radiographs and Clinical Data from the Study of Osteoporotic Fractures}, author = {Koser, Niklas C. and Finck, Marten J. and Jan{\ss}en, Silja and Mouton, Coenraad and Lui, Li-Y. and Cummings, Steven R. and K{\"o}ser, Kevin and H{\"o}vener, Jan-B. and Pirk, S{\"o}ren and Gl{\"u}er, Claus-C.}, booktitle = {Proceedings of The 9th International Conference on Medical Imaging with Deep Learning}, pages = {2640--2665}, year = {2026}, editor = {Huo, Yuankai and Gao, Mingchen and Kuo, Chang-Fu and Jin, Yueming and Deng, Ruining}, volume = {315}, series = {Proceedings of Machine Learning Research}, month = {08--10 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v315/main/assets/koser26a/koser26a.pdf}, url = {https://proceedings.mlr.press/v315/koser26a.html}, abstract = {Osteoporotic changes in the hip structure render the proximal femur particularly vulnerable to fractures, which leads to severe consequences for patients’ health and significant socioeconomic burdens, a strongly increasing problem in aging populations. Accurate risk estimation is therefore essential for initiating timely preventive measures. However, the current clinical standard measures bone mineral density (BMD) and the Fracture Risk Assessment Tool (FRAX) provide only limited predictive value. Neither BMD nor FRAX capture structural characteristics that could be derived from pelvic radiographs that are widely available. To address this gap, we present the Attention-to-Survival Fusion (ATSF) model, a multimodal survival analysis framework that combines clinical risk factors (CRFs) with pelvic radiograph features. An attention-based architecture equipped with a deep conditional transformation model (DCTM) prediction head enables accurate estimation of time-dependent fracture risk. The ATSF model is designed to accommodate missing clinical variables, handle all forms of non-informative censoring, and provides modality-specific interpretability through the attention mechanisms. It was developed, validated and tested with data of 7825 women from the Study of Osteoporotic Fractures (SOF) followed for fracture incidence for 23 years. We benchmark ATSF against established baselines, including FRAX, the Cox proportional hazards model (CoxPH), and a deep learning reference model. Our results demonstrate significant superior performance across concordance index (C-index) and area under the receiver operating characteristic curve (AUC), indicating the importance of integrating radiographic and clinical data within a unified survival framework. Furthermore, offering improved interpretability and a scalable multimodal design, the proposed method provides a promising alternative for advancing individualized hip-fracture risk prediction in osteoporosis research and precision medicine.} }
Endnote
%0 Conference Paper %T Attention-to-Survival: Multimodal Fracture Risk Prediction Based on Pelvic Radiographs and Clinical Data from the Study of Osteoporotic Fractures %A Niklas C. Koser %A Marten J. Finck %A Silja Janßen %A Coenraad Mouton %A Li-Y. Lui %A Steven R. Cummings %A Kevin Köser %A Jan-B. Hövener %A Sören Pirk %A Claus-C. Glüer %B Proceedings of The 9th International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2026 %E Yuankai Huo %E Mingchen Gao %E Chang-Fu Kuo %E Yueming Jin %E Ruining Deng %F pmlr-v315-koser26a %I PMLR %P 2640--2665 %U https://proceedings.mlr.press/v315/koser26a.html %V 315 %X Osteoporotic changes in the hip structure render the proximal femur particularly vulnerable to fractures, which leads to severe consequences for patients’ health and significant socioeconomic burdens, a strongly increasing problem in aging populations. Accurate risk estimation is therefore essential for initiating timely preventive measures. However, the current clinical standard measures bone mineral density (BMD) and the Fracture Risk Assessment Tool (FRAX) provide only limited predictive value. Neither BMD nor FRAX capture structural characteristics that could be derived from pelvic radiographs that are widely available. To address this gap, we present the Attention-to-Survival Fusion (ATSF) model, a multimodal survival analysis framework that combines clinical risk factors (CRFs) with pelvic radiograph features. An attention-based architecture equipped with a deep conditional transformation model (DCTM) prediction head enables accurate estimation of time-dependent fracture risk. The ATSF model is designed to accommodate missing clinical variables, handle all forms of non-informative censoring, and provides modality-specific interpretability through the attention mechanisms. It was developed, validated and tested with data of 7825 women from the Study of Osteoporotic Fractures (SOF) followed for fracture incidence for 23 years. We benchmark ATSF against established baselines, including FRAX, the Cox proportional hazards model (CoxPH), and a deep learning reference model. Our results demonstrate significant superior performance across concordance index (C-index) and area under the receiver operating characteristic curve (AUC), indicating the importance of integrating radiographic and clinical data within a unified survival framework. Furthermore, offering improved interpretability and a scalable multimodal design, the proposed method provides a promising alternative for advancing individualized hip-fracture risk prediction in osteoporosis research and precision medicine.
APA
Koser, N.C., Finck, M.J., Janßen, S., Mouton, C., Lui, L., Cummings, S.R., Köser, K., Hövener, J., Pirk, S. & Glüer, C.. (2026). Attention-to-Survival: Multimodal Fracture Risk Prediction Based on Pelvic Radiographs and Clinical Data from the Study of Osteoporotic Fractures. Proceedings of The 9th International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 315:2640-2665 Available from https://proceedings.mlr.press/v315/koser26a.html.

Related Material