Automatic 3D/2D Deformable Registration in Minimally Invasive Liver Resection using a Mesh Recovery Network

Mathieu Labrunie, Daniel Pizarro, Christophe Tilmant, Adrien Bartoli
Medical Imaging with Deep Learning, PMLR 227:1104-1123, 2024.

Abstract

We propose the patient-specific Liver Mesh Recovery (LMR) framework, to automatically achieve Augmented Reality (AR) guidance by registering a preoperative 3D model in Minimally Invasive Liver Resection (MILR). Existing methods solve registration in MILR by pose estimation followed with numerical optimisation and suffer from a prohibitive intraoperative runtime. The proposed LMR is inspired by the recent Human Mesh Recovery (HMR) framework and forms the first learning-based method to solve registration in MILR. In contrast to existing methods, the computation load in LMR occurs preoperatively, at training time. We construct a patient-specific deformation model and generate patient-specific training data reproducing the typical defects of the automatically detected registration primitives. Experimental results show that LMR’s registration accuracy is on par with optimisation-based methods, whilst running in real-time intraoperatively.

Cite this Paper

BibTeX
@InProceedings{pmlr-v227-labrunie24a, title = {Automatic 3D/2D Deformable Registration in Minimally Invasive Liver Resection using a Mesh Recovery Network}, author = {Labrunie, Mathieu and Pizarro, Daniel and Tilmant, Christophe and Bartoli, Adrien}, booktitle = {Medical Imaging with Deep Learning}, pages = {1104--1123}, year = {2024}, editor = {Oguz, Ipek and Noble, Jack and Li, Xiaoxiao and Styner, Martin and Baumgartner, Christian and Rusu, Mirabela and Heinmann, Tobias and Kontos, Despina and Landman, Bennett and Dawant, Benoit}, volume = {227}, series = {Proceedings of Machine Learning Research}, month = {10--12 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v227/labrunie24a/labrunie24a.pdf}, url = {https://proceedings.mlr.press/v227/labrunie24a.html}, abstract = {We propose the patient-specific Liver Mesh Recovery (LMR) framework, to automatically achieve Augmented Reality (AR) guidance by registering a preoperative 3D model in Minimally Invasive Liver Resection (MILR). Existing methods solve registration in MILR by pose estimation followed with numerical optimisation and suffer from a prohibitive intraoperative runtime. The proposed LMR is inspired by the recent Human Mesh Recovery (HMR) framework and forms the first learning-based method to solve registration in MILR. In contrast to existing methods, the computation load in LMR occurs preoperatively, at training time. We construct a patient-specific deformation model and generate patient-specific training data reproducing the typical defects of the automatically detected registration primitives. Experimental results show that LMR’s registration accuracy is on par with optimisation-based methods, whilst running in real-time intraoperatively.} }
Endnote
%0 Conference Paper %T Automatic 3D/2D Deformable Registration in Minimally Invasive Liver Resection using a Mesh Recovery Network %A Mathieu Labrunie %A Daniel Pizarro %A Christophe Tilmant %A Adrien Bartoli %B Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2024 %E Ipek Oguz %E Jack Noble %E Xiaoxiao Li %E Martin Styner %E Christian Baumgartner %E Mirabela Rusu %E Tobias Heinmann %E Despina Kontos %E Bennett Landman %E Benoit Dawant %F pmlr-v227-labrunie24a %I PMLR %P 1104--1123 %U https://proceedings.mlr.press/v227/labrunie24a.html %V 227 %X We propose the patient-specific Liver Mesh Recovery (LMR) framework, to automatically achieve Augmented Reality (AR) guidance by registering a preoperative 3D model in Minimally Invasive Liver Resection (MILR). Existing methods solve registration in MILR by pose estimation followed with numerical optimisation and suffer from a prohibitive intraoperative runtime. The proposed LMR is inspired by the recent Human Mesh Recovery (HMR) framework and forms the first learning-based method to solve registration in MILR. In contrast to existing methods, the computation load in LMR occurs preoperatively, at training time. We construct a patient-specific deformation model and generate patient-specific training data reproducing the typical defects of the automatically detected registration primitives. Experimental results show that LMR’s registration accuracy is on par with optimisation-based methods, whilst running in real-time intraoperatively.
APA
Labrunie, M., Pizarro, D., Tilmant, C. & Bartoli, A.. (2024). Automatic 3D/2D Deformable Registration in Minimally Invasive Liver Resection using a Mesh Recovery Network. Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 227:1104-1123 Available from https://proceedings.mlr.press/v227/labrunie24a.html.

Related Material