Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning

Milan Ganai, Rohan Sinha, Christopher Agia, Daniel Morton, Luigi Di Lillo, Marco Pavone
Proceedings of The 9th Conference on Robot Learning, PMLR 305:283-308, 2025.

Abstract

Foundation models can provide robust high-level reasoning on appropriate safety interventions in hazardous scenarios beyond a robot’s training data, i.e. out-of-distribution (OOD) failures. However, due to the high inference latency of Large Vision and Language Models, current methods rely on manually defined intervention policies to enact fallbacks, thereby lacking the ability to plan generalizable, semantically safe motions. To overcome these challenges we present FORTRESS, a framework that generates and reasons about semantically safe fallback strategies in real time to prevent OOD failures. At a low frequency in nominal operations, FORTRESS uses multi-modal reasoners to identify goals and anticipate failure modes. When a runtime monitor triggers a fallback response, FORTRESS rapidly synthesizes plans to fallback goals while inferring and avoiding semantically unsafe regions in real time. By bridging open-world, multi-modal reasoning with dynamics-aware planning, we eliminate the need for hard-coded fallbacks and human safety interventions. FORTRESS outperforms on-the-fly prompting of slow reasoning models in safety classification accuracy on synthetic benchmarks and real-world ANYmal robot data, and further improves system safety and planning success in simulation and on quadrotor hardware for urban navigation. Website and code can be found at https://submfort.github.io/fortress/.

Cite this Paper

BibTeX
@InProceedings{pmlr-v305-ganai25a, title = {Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning}, author = {Ganai, Milan and Sinha, Rohan and Agia, Christopher and Morton, Daniel and Lillo, Luigi Di and Pavone, Marco}, booktitle = {Proceedings of The 9th Conference on Robot Learning}, pages = {283--308}, year = {2025}, editor = {Lim, Joseph and Song, Shuran and Park, Hae-Won}, volume = {305}, series = {Proceedings of Machine Learning Research}, month = {27--30 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v305/main/assets/ganai25a/ganai25a.pdf}, url = {https://proceedings.mlr.press/v305/ganai25a.html}, abstract = {Foundation models can provide robust high-level reasoning on appropriate safety interventions in hazardous scenarios beyond a robot’s training data, i.e. out-of-distribution (OOD) failures. However, due to the high inference latency of Large Vision and Language Models, current methods rely on manually defined intervention policies to enact fallbacks, thereby lacking the ability to plan generalizable, semantically safe motions. To overcome these challenges we present FORTRESS, a framework that generates and reasons about semantically safe fallback strategies in real time to prevent OOD failures. At a low frequency in nominal operations, FORTRESS uses multi-modal reasoners to identify goals and anticipate failure modes. When a runtime monitor triggers a fallback response, FORTRESS rapidly synthesizes plans to fallback goals while inferring and avoiding semantically unsafe regions in real time. By bridging open-world, multi-modal reasoning with dynamics-aware planning, we eliminate the need for hard-coded fallbacks and human safety interventions. FORTRESS outperforms on-the-fly prompting of slow reasoning models in safety classification accuracy on synthetic benchmarks and real-world ANYmal robot data, and further improves system safety and planning success in simulation and on quadrotor hardware for urban navigation. Website and code can be found at https://submfort.github.io/fortress/.} }
Endnote
%0 Conference Paper %T Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning %A Milan Ganai %A Rohan Sinha %A Christopher Agia %A Daniel Morton %A Luigi Di Lillo %A Marco Pavone %B Proceedings of The 9th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Joseph Lim %E Shuran Song %E Hae-Won Park %F pmlr-v305-ganai25a %I PMLR %P 283--308 %U https://proceedings.mlr.press/v305/ganai25a.html %V 305 %X Foundation models can provide robust high-level reasoning on appropriate safety interventions in hazardous scenarios beyond a robot’s training data, i.e. out-of-distribution (OOD) failures. However, due to the high inference latency of Large Vision and Language Models, current methods rely on manually defined intervention policies to enact fallbacks, thereby lacking the ability to plan generalizable, semantically safe motions. To overcome these challenges we present FORTRESS, a framework that generates and reasons about semantically safe fallback strategies in real time to prevent OOD failures. At a low frequency in nominal operations, FORTRESS uses multi-modal reasoners to identify goals and anticipate failure modes. When a runtime monitor triggers a fallback response, FORTRESS rapidly synthesizes plans to fallback goals while inferring and avoiding semantically unsafe regions in real time. By bridging open-world, multi-modal reasoning with dynamics-aware planning, we eliminate the need for hard-coded fallbacks and human safety interventions. FORTRESS outperforms on-the-fly prompting of slow reasoning models in safety classification accuracy on synthetic benchmarks and real-world ANYmal robot data, and further improves system safety and planning success in simulation and on quadrotor hardware for urban navigation. Website and code can be found at https://submfort.github.io/fortress/.
APA
Ganai, M., Sinha, R., Agia, C., Morton, D., Lillo, L.D. & Pavone, M.. (2025). Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning. Proceedings of The 9th Conference on Robot Learning, in Proceedings of Machine Learning Research 305:283-308 Available from https://proceedings.mlr.press/v305/ganai25a.html.

Related Material