REINDIR: Repeated Embedding Infusion for Neural Deformable Image Registration

Louis van Harten, Rudolf Leonardus Mirjam Van Herten, Ivana Isgum
Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, PMLR 250:577-595, 2024.

Abstract

The use of implicit neural representations (INRs) has been explored for medical image registration in a number of recent works. Using these representations has several advantages over both classic optimization-based methods and deep learning-based methods, but it is hindered by long optimization times during inference. To address this issue, we propose REINDIR: Repeated Embedding Infusion for Neural Deformable Image Registration. REINDIR is a meta-learning framework that uses a combination of an image encoder and template representations, which are infused with image embeddings to specialize them for a pair of test images. This specialization results in a better initialization for the subsequent optimization process. By broadcasting the encodings to fill our modulation weight matrices, we greatly reduce the required size of the encoder compared to approaches that predict the complete weight matrices directly. Additionally, our method retains the flexibility to infuse arbitrarily large encodings. The presented approach greatly improves the efficiency of deformable registration with INRs when applied to (near-)IID data, while remaining robust to severe domain shifts from the distribution the method is trained on.

Cite this Paper

BibTeX
@InProceedings{pmlr-v250-harten24a, title = {REINDIR: Repeated Embedding Infusion for Neural Deformable Image Registration}, author = {van Harten, Louis and Herten, Rudolf Leonardus Mirjam Van and Isgum, Ivana}, booktitle = {Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning}, pages = {577--595}, year = {2024}, editor = {Burgos, Ninon and Petitjean, Caroline and Vakalopoulou, Maria and Christodoulidis, Stergios and Coupe, Pierrick and Delingette, Hervé and Lartizien, Carole and Mateus, Diana}, volume = {250}, series = {Proceedings of Machine Learning Research}, month = {03--05 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v250/main/assets/harten24a/harten24a.pdf}, url = {https://proceedings.mlr.press/v250/harten24a.html}, abstract = {The use of implicit neural representations (INRs) has been explored for medical image registration in a number of recent works. Using these representations has several advantages over both classic optimization-based methods and deep learning-based methods, but it is hindered by long optimization times during inference. To address this issue, we propose REINDIR: Repeated Embedding Infusion for Neural Deformable Image Registration. REINDIR is a meta-learning framework that uses a combination of an image encoder and template representations, which are infused with image embeddings to specialize them for a pair of test images. This specialization results in a better initialization for the subsequent optimization process. By broadcasting the encodings to fill our modulation weight matrices, we greatly reduce the required size of the encoder compared to approaches that predict the complete weight matrices directly. Additionally, our method retains the flexibility to infuse arbitrarily large encodings. The presented approach greatly improves the efficiency of deformable registration with INRs when applied to (near-)IID data, while remaining robust to severe domain shifts from the distribution the method is trained on.} }
Endnote
%0 Conference Paper %T REINDIR: Repeated Embedding Infusion for Neural Deformable Image Registration %A Louis van Harten %A Rudolf Leonardus Mirjam Van Herten %A Ivana Isgum %B Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2024 %E Ninon Burgos %E Caroline Petitjean %E Maria Vakalopoulou %E Stergios Christodoulidis %E Pierrick Coupe %E Hervé Delingette %E Carole Lartizien %E Diana Mateus %F pmlr-v250-harten24a %I PMLR %P 577--595 %U https://proceedings.mlr.press/v250/harten24a.html %V 250 %X The use of implicit neural representations (INRs) has been explored for medical image registration in a number of recent works. Using these representations has several advantages over both classic optimization-based methods and deep learning-based methods, but it is hindered by long optimization times during inference. To address this issue, we propose REINDIR: Repeated Embedding Infusion for Neural Deformable Image Registration. REINDIR is a meta-learning framework that uses a combination of an image encoder and template representations, which are infused with image embeddings to specialize them for a pair of test images. This specialization results in a better initialization for the subsequent optimization process. By broadcasting the encodings to fill our modulation weight matrices, we greatly reduce the required size of the encoder compared to approaches that predict the complete weight matrices directly. Additionally, our method retains the flexibility to infuse arbitrarily large encodings. The presented approach greatly improves the efficiency of deformable registration with INRs when applied to (near-)IID data, while remaining robust to severe domain shifts from the distribution the method is trained on.
APA
van Harten, L., Herten, R.L.M.V. & Isgum, I.. (2024). REINDIR: Repeated Embedding Infusion for Neural Deformable Image Registration. Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 250:577-595 Available from https://proceedings.mlr.press/v250/harten24a.html.

Related Material