DVI:A Derivative-based Vision Network for INR

Runzhao Yang, Xiaolong Wu, Zhihong Zhang, Fabian Zhang, Tingxiong Xiao, Zongren Li, Kunlun He, Jinli Suo
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:71405-71424, 2025.

Abstract

Recent advancements in computer vision have seen Implicit Neural Representations (INR) becoming a dominant representation form for data due to their compactness and expressive power. To solve various vision tasks with INR data, vision networks can either be purely INR-based, but are thereby limited by simplistic operations and performance constraints, or include raster-based methods, which then tend to lose crucial structural information of the INR during the conversion process. To address these issues, we propose DVI, a novel Derivative-based Vision network for INR, capable of handling a variety of vision tasks across various data modalities, while achieving the best performance among the existing methods by incorporating state of the art raster-based methods into a INR based architecture. DVI excels by extracting semantic information from the high order derivative map of the INR, then seamlessly fusing it into a pre-existing raster-based vision network, enhancing its performance with deeper, task-relevant semantic insights. Extensive experiments on five vision tasks across three data modalities demonstrate DVI’s superiority over existing methods. Additionally, our study encompasses comprehensive ablation studies to affirm the efficacy of each element of DVI, the influence of different derivative computation techniques and the impact of derivative orders. Reproducible codes are provided in the supplementary materials.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-yang25an, title = {{DVI}:{A} Derivative-based Vision Network for {INR}}, author = {Yang, Runzhao and Wu, Xiaolong and Zhang, Zhihong and Zhang, Fabian and Xiao, Tingxiong and Li, Zongren and He, Kunlun and Suo, Jinli}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {71405--71424}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/yang25an/yang25an.pdf}, url = {https://proceedings.mlr.press/v267/yang25an.html}, abstract = {Recent advancements in computer vision have seen Implicit Neural Representations (INR) becoming a dominant representation form for data due to their compactness and expressive power. To solve various vision tasks with INR data, vision networks can either be purely INR-based, but are thereby limited by simplistic operations and performance constraints, or include raster-based methods, which then tend to lose crucial structural information of the INR during the conversion process. To address these issues, we propose DVI, a novel Derivative-based Vision network for INR, capable of handling a variety of vision tasks across various data modalities, while achieving the best performance among the existing methods by incorporating state of the art raster-based methods into a INR based architecture. DVI excels by extracting semantic information from the high order derivative map of the INR, then seamlessly fusing it into a pre-existing raster-based vision network, enhancing its performance with deeper, task-relevant semantic insights. Extensive experiments on five vision tasks across three data modalities demonstrate DVI’s superiority over existing methods. Additionally, our study encompasses comprehensive ablation studies to affirm the efficacy of each element of DVI, the influence of different derivative computation techniques and the impact of derivative orders. Reproducible codes are provided in the supplementary materials.} }
Endnote
%0 Conference Paper %T DVI:A Derivative-based Vision Network for INR %A Runzhao Yang %A Xiaolong Wu %A Zhihong Zhang %A Fabian Zhang %A Tingxiong Xiao %A Zongren Li %A Kunlun He %A Jinli Suo %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-yang25an %I PMLR %P 71405--71424 %U https://proceedings.mlr.press/v267/yang25an.html %V 267 %X Recent advancements in computer vision have seen Implicit Neural Representations (INR) becoming a dominant representation form for data due to their compactness and expressive power. To solve various vision tasks with INR data, vision networks can either be purely INR-based, but are thereby limited by simplistic operations and performance constraints, or include raster-based methods, which then tend to lose crucial structural information of the INR during the conversion process. To address these issues, we propose DVI, a novel Derivative-based Vision network for INR, capable of handling a variety of vision tasks across various data modalities, while achieving the best performance among the existing methods by incorporating state of the art raster-based methods into a INR based architecture. DVI excels by extracting semantic information from the high order derivative map of the INR, then seamlessly fusing it into a pre-existing raster-based vision network, enhancing its performance with deeper, task-relevant semantic insights. Extensive experiments on five vision tasks across three data modalities demonstrate DVI’s superiority over existing methods. Additionally, our study encompasses comprehensive ablation studies to affirm the efficacy of each element of DVI, the influence of different derivative computation techniques and the impact of derivative orders. Reproducible codes are provided in the supplementary materials.
APA
Yang, R., Wu, X., Zhang, Z., Zhang, F., Xiao, T., Li, Z., He, K. & Suo, J.. (2025). DVI:A Derivative-based Vision Network for INR. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:71405-71424 Available from https://proceedings.mlr.press/v267/yang25an.html.

Related Material