SG2Loc: Sequential Visual Localization on 3D Scene Graphs

Nicole Damblon, Olga Vysotska, Federico Tombari, Marc Pollefeys, Daniel Barath
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:22729-22742, 2026.

Abstract

Visual localization in complex indoor environments remains a critical challenge for robotics and AR applications. Sequential localization, where pose estimates are refined over time, is important for autonomous agents. However, traditional methods often require storing extensive image databases or point clouds, leading to significant overhead. This paper introduces a novel, lightweight approach to sequential visual localization using 3D scene graphs. Our method represents the environment with a compact scene graph, where nodes represent objects (with coarse meshes) and edges encode spatial relationships. For each image in the localization phase, we extract per-patch semantic features, predicting object identities. Localization is performed within a particle filter framework. Each particle, representing a camera pose, projects the coarse object meshes from the scene graph into the image, assigning object identities to patches based on visibility. The similarity of the per-patch features, in the input image, and object features from the scene graph determines the weight of a particle. Subsequent images are incorporated sequentially, refining the pose estimate. By leveraging a compact scene graph and efficient semantic matching, our method significantly reduces storage while maintaining performance on real-world datasets. The code is available at https://github.com/DmblnNicole/sg2loc.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-damblon26a, title = {{SG}2{L}oc: Sequential Visual Localization on 3{D} Scene Graphs}, author = {Damblon, Nicole and Vysotska, Olga and Tombari, Federico and Pollefeys, Marc and Barath, Daniel}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {22729--22742}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/damblon26a/damblon26a.pdf}, url = {https://proceedings.mlr.press/v306/damblon26a.html}, abstract = {Visual localization in complex indoor environments remains a critical challenge for robotics and AR applications. Sequential localization, where pose estimates are refined over time, is important for autonomous agents. However, traditional methods often require storing extensive image databases or point clouds, leading to significant overhead. This paper introduces a novel, lightweight approach to sequential visual localization using 3D scene graphs. Our method represents the environment with a compact scene graph, where nodes represent objects (with coarse meshes) and edges encode spatial relationships. For each image in the localization phase, we extract per-patch semantic features, predicting object identities. Localization is performed within a particle filter framework. Each particle, representing a camera pose, projects the coarse object meshes from the scene graph into the image, assigning object identities to patches based on visibility. The similarity of the per-patch features, in the input image, and object features from the scene graph determines the weight of a particle. Subsequent images are incorporated sequentially, refining the pose estimate. By leveraging a compact scene graph and efficient semantic matching, our method significantly reduces storage while maintaining performance on real-world datasets. The code is available at https://github.com/DmblnNicole/sg2loc.} }
Endnote
%0 Conference Paper %T SG2Loc: Sequential Visual Localization on 3D Scene Graphs %A Nicole Damblon %A Olga Vysotska %A Federico Tombari %A Marc Pollefeys %A Daniel Barath %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-damblon26a %I PMLR %P 22729--22742 %U https://proceedings.mlr.press/v306/damblon26a.html %V 306 %X Visual localization in complex indoor environments remains a critical challenge for robotics and AR applications. Sequential localization, where pose estimates are refined over time, is important for autonomous agents. However, traditional methods often require storing extensive image databases or point clouds, leading to significant overhead. This paper introduces a novel, lightweight approach to sequential visual localization using 3D scene graphs. Our method represents the environment with a compact scene graph, where nodes represent objects (with coarse meshes) and edges encode spatial relationships. For each image in the localization phase, we extract per-patch semantic features, predicting object identities. Localization is performed within a particle filter framework. Each particle, representing a camera pose, projects the coarse object meshes from the scene graph into the image, assigning object identities to patches based on visibility. The similarity of the per-patch features, in the input image, and object features from the scene graph determines the weight of a particle. Subsequent images are incorporated sequentially, refining the pose estimate. By leveraging a compact scene graph and efficient semantic matching, our method significantly reduces storage while maintaining performance on real-world datasets. The code is available at https://github.com/DmblnNicole/sg2loc.
APA
Damblon, N., Vysotska, O., Tombari, F., Pollefeys, M. & Barath, D.. (2026). SG2Loc: Sequential Visual Localization on 3D Scene Graphs. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:22729-22742 Available from https://proceedings.mlr.press/v306/damblon26a.html.

Related Material