SELFI: Autonomous Self-Improvement with RL for Vision-Based Navigation around People

Noriaki Hirose, Dhruv Shah, Kyle Stachowicz, Ajay Sridhar, Sergey Levine
Proceedings of The 8th Conference on Robot Learning, PMLR 270:97-116, 2025.

Abstract

Autonomous self-improving robots that interact and improve with experience are key to the real-world deployment of robotic systems. In this paper, we propose an online learning method, SELFI, that leverages online robot experience to rapidly fine-tune pre-trained control policies efficiently. SELFI applies online model-free reinforcement learning on top of offline model-based learning to bring out the best parts of both learning paradigms. Specifically, SELFI stabilizes the online learning process by incorporating the same model-based learning objective from offline pre-training into the Q-values learned with online model-free reinforcement learning. We evaluate SELFI in multiple real-world environments and report improvements in terms of collision avoidance, as well as more socially compliant behavior, measured by a human user study. SELFI enables us to quickly learn useful robotic behaviors with less human interventions such as pre-emptive behavior for the pedestrians, collision avoidance for small and transparent objects, and avoiding travel on uneven floor surfaces. We provide supplementary videos to demonstrate the performance of our fine-tuned policy.

Cite this Paper

BibTeX
@InProceedings{pmlr-v270-hirose25a, title = {SELFI: Autonomous Self-Improvement with RL for Vision-Based Navigation around People}, author = {Hirose, Noriaki and Shah, Dhruv and Stachowicz, Kyle and Sridhar, Ajay and Levine, Sergey}, booktitle = {Proceedings of The 8th Conference on Robot Learning}, pages = {97--116}, year = {2025}, editor = {Agrawal, Pulkit and Kroemer, Oliver and Burgard, Wolfram}, volume = {270}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v270/main/assets/hirose25a/hirose25a.pdf}, url = {https://proceedings.mlr.press/v270/hirose25a.html}, abstract = {Autonomous self-improving robots that interact and improve with experience are key to the real-world deployment of robotic systems. In this paper, we propose an online learning method, SELFI, that leverages online robot experience to rapidly fine-tune pre-trained control policies efficiently. SELFI applies online model-free reinforcement learning on top of offline model-based learning to bring out the best parts of both learning paradigms. Specifically, SELFI stabilizes the online learning process by incorporating the same model-based learning objective from offline pre-training into the Q-values learned with online model-free reinforcement learning. We evaluate SELFI in multiple real-world environments and report improvements in terms of collision avoidance, as well as more socially compliant behavior, measured by a human user study. SELFI enables us to quickly learn useful robotic behaviors with less human interventions such as pre-emptive behavior for the pedestrians, collision avoidance for small and transparent objects, and avoiding travel on uneven floor surfaces. We provide supplementary videos to demonstrate the performance of our fine-tuned policy.} }
Endnote
%0 Conference Paper %T SELFI: Autonomous Self-Improvement with RL for Vision-Based Navigation around People %A Noriaki Hirose %A Dhruv Shah %A Kyle Stachowicz %A Ajay Sridhar %A Sergey Levine %B Proceedings of The 8th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Pulkit Agrawal %E Oliver Kroemer %E Wolfram Burgard %F pmlr-v270-hirose25a %I PMLR %P 97--116 %U https://proceedings.mlr.press/v270/hirose25a.html %V 270 %X Autonomous self-improving robots that interact and improve with experience are key to the real-world deployment of robotic systems. In this paper, we propose an online learning method, SELFI, that leverages online robot experience to rapidly fine-tune pre-trained control policies efficiently. SELFI applies online model-free reinforcement learning on top of offline model-based learning to bring out the best parts of both learning paradigms. Specifically, SELFI stabilizes the online learning process by incorporating the same model-based learning objective from offline pre-training into the Q-values learned with online model-free reinforcement learning. We evaluate SELFI in multiple real-world environments and report improvements in terms of collision avoidance, as well as more socially compliant behavior, measured by a human user study. SELFI enables us to quickly learn useful robotic behaviors with less human interventions such as pre-emptive behavior for the pedestrians, collision avoidance for small and transparent objects, and avoiding travel on uneven floor surfaces. We provide supplementary videos to demonstrate the performance of our fine-tuned policy.
APA
Hirose, N., Shah, D., Stachowicz, K., Sridhar, A. & Levine, S.. (2025). SELFI: Autonomous Self-Improvement with RL for Vision-Based Navigation around People. Proceedings of The 8th Conference on Robot Learning, in Proceedings of Machine Learning Research 270:97-116 Available from https://proceedings.mlr.press/v270/hirose25a.html.

Related Material