Self-Improving Semantic Perception for Indoor Localisation

Hermann Blum, Francesco Milano, René Zurbrügg, Roland Siegwart, Cesar Cadena, Abel Gawel
Proceedings of the 5th Conference on Robot Learning, PMLR 164:1211-1222, 2022.

Abstract

We propose a novel robotic system that can improve its perception during deployment. Contrary to the established approach of learning semantics from large datasets and deploying fixed models, we propose a framework in which semantic models are continuously updated on the robot to adapt to the deployment environments. By combining continual learning with self-supervision, our robotic system learns online during deployment without external supervision. We conduct real-world experiments with robots localising in 3D floorplans. Our experiments show how the robot’s semantic perception improves during deployment and how this translates into improved localisation, even across drastically different environments. We further study the risk of catastrophic forgetting that such a continuous learning setting poses. We find memory replay an effective measure to reduce forgetting and show how the robotic system can improve even when switching between different environments. On average, our system improves by 60% in segmentation and 10% in localisation accuracy compared to deployment of a fixed model, and it maintains this improvement while adapting to further environments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-blum22a, title = {Self-Improving Semantic Perception for Indoor Localisation}, author = {Blum, Hermann and Milano, Francesco and Zurbr\"ugg, Ren\'e and Siegwart, Roland and Cadena, Cesar and Gawel, Abel}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {1211--1222}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/blum22a/blum22a.pdf}, url = {https://proceedings.mlr.press/v164/blum22a.html}, abstract = {We propose a novel robotic system that can improve its perception during deployment. Contrary to the established approach of learning semantics from large datasets and deploying fixed models, we propose a framework in which semantic models are continuously updated on the robot to adapt to the deployment environments. By combining continual learning with self-supervision, our robotic system learns online during deployment without external supervision. We conduct real-world experiments with robots localising in 3D floorplans. Our experiments show how the robot’s semantic perception improves during deployment and how this translates into improved localisation, even across drastically different environments. We further study the risk of catastrophic forgetting that such a continuous learning setting poses. We find memory replay an effective measure to reduce forgetting and show how the robotic system can improve even when switching between different environments. On average, our system improves by 60% in segmentation and 10% in localisation accuracy compared to deployment of a fixed model, and it maintains this improvement while adapting to further environments.} }
Endnote
%0 Conference Paper %T Self-Improving Semantic Perception for Indoor Localisation %A Hermann Blum %A Francesco Milano %A René Zurbrügg %A Roland Siegwart %A Cesar Cadena %A Abel Gawel %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-blum22a %I PMLR %P 1211--1222 %U https://proceedings.mlr.press/v164/blum22a.html %V 164 %X We propose a novel robotic system that can improve its perception during deployment. Contrary to the established approach of learning semantics from large datasets and deploying fixed models, we propose a framework in which semantic models are continuously updated on the robot to adapt to the deployment environments. By combining continual learning with self-supervision, our robotic system learns online during deployment without external supervision. We conduct real-world experiments with robots localising in 3D floorplans. Our experiments show how the robot’s semantic perception improves during deployment and how this translates into improved localisation, even across drastically different environments. We further study the risk of catastrophic forgetting that such a continuous learning setting poses. We find memory replay an effective measure to reduce forgetting and show how the robotic system can improve even when switching between different environments. On average, our system improves by 60% in segmentation and 10% in localisation accuracy compared to deployment of a fixed model, and it maintains this improvement while adapting to further environments.
APA
Blum, H., Milano, F., Zurbrügg, R., Siegwart, R., Cadena, C. & Gawel, A.. (2022). Self-Improving Semantic Perception for Indoor Localisation. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:1211-1222 Available from https://proceedings.mlr.press/v164/blum22a.html.

Related Material