PRISM: Probabilistic Real-Time Inference in Spatial World Models

Atanas Mirchev, Baris Kayalibay, Ahmed Agha, Patrick van der Smagt, Daniel Cremers, Justin Bayer
Proceedings of The 6th Conference on Robot Learning, PMLR 205:161-174, 2023.

Abstract

We introduce PRISM, a method for real-time filtering in a probabilistic generative model of agent motion and visual perception. Previous approaches either lack uncertainty estimates for the map and agent state, do not run in real-time, do not have a dense scene representation or do not model agent dynamics. Our solution reconciles all of these aspects. We start from a predefined state-space model which combines differentiable rendering and 6-DoF dynamics. Probabilistic inference in this model amounts to simultaneous localisation and mapping (SLAM) and is intractable. We use a series of approximations to Bayesian inference to arrive at probabilistic map and state estimates. We take advantage of well-established methods and closed-form updates, preserving accuracy and enabling real-time capability. The proposed solution runs at 10Hz real-time and is similarly accurate to state-of-the-art SLAM in small to medium-sized indoor environments, with high-speed UAV and handheld camera agents (Blackbird, EuRoC and TUM-RGBD).

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-mirchev23a, title = {PRISM: Probabilistic Real-Time Inference in Spatial World Models}, author = {Mirchev, Atanas and Kayalibay, Baris and Agha, Ahmed and Smagt, Patrick van der and Cremers, Daniel and Bayer, Justin}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {161--174}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/mirchev23a/mirchev23a.pdf}, url = {https://proceedings.mlr.press/v205/mirchev23a.html}, abstract = {We introduce PRISM, a method for real-time filtering in a probabilistic generative model of agent motion and visual perception. Previous approaches either lack uncertainty estimates for the map and agent state, do not run in real-time, do not have a dense scene representation or do not model agent dynamics. Our solution reconciles all of these aspects. We start from a predefined state-space model which combines differentiable rendering and 6-DoF dynamics. Probabilistic inference in this model amounts to simultaneous localisation and mapping (SLAM) and is intractable. We use a series of approximations to Bayesian inference to arrive at probabilistic map and state estimates. We take advantage of well-established methods and closed-form updates, preserving accuracy and enabling real-time capability. The proposed solution runs at 10Hz real-time and is similarly accurate to state-of-the-art SLAM in small to medium-sized indoor environments, with high-speed UAV and handheld camera agents (Blackbird, EuRoC and TUM-RGBD). } }
Endnote
%0 Conference Paper %T PRISM: Probabilistic Real-Time Inference in Spatial World Models %A Atanas Mirchev %A Baris Kayalibay %A Ahmed Agha %A Patrick van der Smagt %A Daniel Cremers %A Justin Bayer %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-mirchev23a %I PMLR %P 161--174 %U https://proceedings.mlr.press/v205/mirchev23a.html %V 205 %X We introduce PRISM, a method for real-time filtering in a probabilistic generative model of agent motion and visual perception. Previous approaches either lack uncertainty estimates for the map and agent state, do not run in real-time, do not have a dense scene representation or do not model agent dynamics. Our solution reconciles all of these aspects. We start from a predefined state-space model which combines differentiable rendering and 6-DoF dynamics. Probabilistic inference in this model amounts to simultaneous localisation and mapping (SLAM) and is intractable. We use a series of approximations to Bayesian inference to arrive at probabilistic map and state estimates. We take advantage of well-established methods and closed-form updates, preserving accuracy and enabling real-time capability. The proposed solution runs at 10Hz real-time and is similarly accurate to state-of-the-art SLAM in small to medium-sized indoor environments, with high-speed UAV and handheld camera agents (Blackbird, EuRoC and TUM-RGBD).
APA
Mirchev, A., Kayalibay, B., Agha, A., Smagt, P.v.d., Cremers, D. & Bayer, J.. (2023). PRISM: Probabilistic Real-Time Inference in Spatial World Models. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:161-174 Available from https://proceedings.mlr.press/v205/mirchev23a.html.

Related Material