Vesselformer: Towards Complete 3D Vessel Graph Generation from Images

Chinmay Prabhakar, Suprosanna Shit, Johannes C. Paetzold, Ivan Ezhov, Rajat Koner, Hongwei Li, Florian Sebastian Kofler, Bjoern Menze
Medical Imaging with Deep Learning, PMLR 227:320-331, 2024.

Abstract

The reconstruction of graph representations from Images (Image-to-graph) is a frequent task, especially vessel graph extraction from biomedical images. Traditionally, this problem is tackled by a two-stage process: segmentation followed by skeletonization. However, the ambiguity in the heuristic-based pruning of the centerline graph from the skeleta makes it hard to achieve a compact yet faithful graph representation. Recently, \textit{Relationformer} proposed an end-to-end solution to extract graphs directly from images. However, it does not consider edge features, particularly radius information, which is crucial in many applications such as flow simulation. Further, Relationformer predicts only patch-based graphs. In this work, we address these two shortcomings. We propose a task-specific token, namely radius-token, which explicitly focuses on capturing radius information between two nodes. Second, we propose an efficient algorithm to infer a large 3D graph from patch inference. Finally, we show experimental results on a synthetic vessel dataset and achieve the first 3D complete graph prediction. Code is available at \url{https://github.com/****}.

Cite this Paper

BibTeX
@InProceedings{pmlr-v227-prabhakar24a, title = {Vesselformer: Towards Complete 3D Vessel Graph Generation from Images}, author = {Prabhakar, Chinmay and Shit, Suprosanna and Paetzold, Johannes C. and Ezhov, Ivan and Koner, Rajat and Li, Hongwei and Kofler, Florian Sebastian and Menze, Bjoern}, booktitle = {Medical Imaging with Deep Learning}, pages = {320--331}, year = {2024}, editor = {Oguz, Ipek and Noble, Jack and Li, Xiaoxiao and Styner, Martin and Baumgartner, Christian and Rusu, Mirabela and Heinmann, Tobias and Kontos, Despina and Landman, Bennett and Dawant, Benoit}, volume = {227}, series = {Proceedings of Machine Learning Research}, month = {10--12 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v227/prabhakar24a/prabhakar24a.pdf}, url = {https://proceedings.mlr.press/v227/prabhakar24a.html}, abstract = {The reconstruction of graph representations from Images (Image-to-graph) is a frequent task, especially vessel graph extraction from biomedical images. Traditionally, this problem is tackled by a two-stage process: segmentation followed by skeletonization. However, the ambiguity in the heuristic-based pruning of the centerline graph from the skeleta makes it hard to achieve a compact yet faithful graph representation. Recently, \textit{Relationformer} proposed an end-to-end solution to extract graphs directly from images. However, it does not consider edge features, particularly radius information, which is crucial in many applications such as flow simulation. Further, Relationformer predicts only patch-based graphs. In this work, we address these two shortcomings. We propose a task-specific token, namely radius-token, which explicitly focuses on capturing radius information between two nodes. Second, we propose an efficient algorithm to infer a large 3D graph from patch inference. Finally, we show experimental results on a synthetic vessel dataset and achieve the first 3D complete graph prediction. Code is available at \url{https://github.com/****}.} }
Endnote
%0 Conference Paper %T Vesselformer: Towards Complete 3D Vessel Graph Generation from Images %A Chinmay Prabhakar %A Suprosanna Shit %A Johannes C. Paetzold %A Ivan Ezhov %A Rajat Koner %A Hongwei Li %A Florian Sebastian Kofler %A Bjoern Menze %B Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2024 %E Ipek Oguz %E Jack Noble %E Xiaoxiao Li %E Martin Styner %E Christian Baumgartner %E Mirabela Rusu %E Tobias Heinmann %E Despina Kontos %E Bennett Landman %E Benoit Dawant %F pmlr-v227-prabhakar24a %I PMLR %P 320--331 %U https://proceedings.mlr.press/v227/prabhakar24a.html %V 227 %X The reconstruction of graph representations from Images (Image-to-graph) is a frequent task, especially vessel graph extraction from biomedical images. Traditionally, this problem is tackled by a two-stage process: segmentation followed by skeletonization. However, the ambiguity in the heuristic-based pruning of the centerline graph from the skeleta makes it hard to achieve a compact yet faithful graph representation. Recently, \textit{Relationformer} proposed an end-to-end solution to extract graphs directly from images. However, it does not consider edge features, particularly radius information, which is crucial in many applications such as flow simulation. Further, Relationformer predicts only patch-based graphs. In this work, we address these two shortcomings. We propose a task-specific token, namely radius-token, which explicitly focuses on capturing radius information between two nodes. Second, we propose an efficient algorithm to infer a large 3D graph from patch inference. Finally, we show experimental results on a synthetic vessel dataset and achieve the first 3D complete graph prediction. Code is available at \url{https://github.com/****}.
APA
Prabhakar, C., Shit, S., Paetzold, J.C., Ezhov, I., Koner, R., Li, H., Kofler, F.S. & Menze, B.. (2024). Vesselformer: Towards Complete 3D Vessel Graph Generation from Images. Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 227:320-331 Available from https://proceedings.mlr.press/v227/prabhakar24a.html.

Related Material