Cost-aware Discovery of Contextual Failures using Bayesian Active Learning

Anjali Parashar, Joseph Zhang, Yingke Li, Chuchu Fan
Proceedings of The 9th Conference on Robot Learning, PMLR 305:2239-2267, 2025.

Abstract

Ensuring the robustness of robotic systems is crucial for their deployment in safety-critical domains. Failure discovery, or falsification, is a widely used approach for evaluating robustness, with recent advancements focusing on improving sample efficiency and generalization through probabilistic sampling techniques and learning-theoretic approaches. However, existing methods typically rely on explicitly defined analytical cost functions to characterize failures, often overlooking the underlying causes and diversity of discovered failure scenarios. In this work, we propose a novel failure discovery framework that integrates contextual reasoning in the falsification process, specifically tailored for high evaluation-cost applications. Our method incorporates expert-in-the-loop feedback to construct a probabilistic surrogate model of failures using Bayesian inference. This model is iteratively refined and leveraged to guide an active learning strategy that prioritizes the discovery of diverse failure cases. We empirically validate our approach across a range of tasks for high-cost contextual falsification in robotic manipulation and autonomous driving.

Cite this Paper

BibTeX
@InProceedings{pmlr-v305-parashar25a, title = {Cost-aware Discovery of Contextual Failures using Bayesian Active Learning}, author = {Parashar, Anjali and Zhang, Joseph and Li, Yingke and Fan, Chuchu}, booktitle = {Proceedings of The 9th Conference on Robot Learning}, pages = {2239--2267}, year = {2025}, editor = {Lim, Joseph and Song, Shuran and Park, Hae-Won}, volume = {305}, series = {Proceedings of Machine Learning Research}, month = {27--30 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v305/main/assets/parashar25a/parashar25a.pdf}, url = {https://proceedings.mlr.press/v305/parashar25a.html}, abstract = {Ensuring the robustness of robotic systems is crucial for their deployment in safety-critical domains. Failure discovery, or falsification, is a widely used approach for evaluating robustness, with recent advancements focusing on improving sample efficiency and generalization through probabilistic sampling techniques and learning-theoretic approaches. However, existing methods typically rely on explicitly defined analytical cost functions to characterize failures, often overlooking the underlying causes and diversity of discovered failure scenarios. In this work, we propose a novel failure discovery framework that integrates contextual reasoning in the falsification process, specifically tailored for high evaluation-cost applications. Our method incorporates expert-in-the-loop feedback to construct a probabilistic surrogate model of failures using Bayesian inference. This model is iteratively refined and leveraged to guide an active learning strategy that prioritizes the discovery of diverse failure cases. We empirically validate our approach across a range of tasks for high-cost contextual falsification in robotic manipulation and autonomous driving.} }
Endnote
%0 Conference Paper %T Cost-aware Discovery of Contextual Failures using Bayesian Active Learning %A Anjali Parashar %A Joseph Zhang %A Yingke Li %A Chuchu Fan %B Proceedings of The 9th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Joseph Lim %E Shuran Song %E Hae-Won Park %F pmlr-v305-parashar25a %I PMLR %P 2239--2267 %U https://proceedings.mlr.press/v305/parashar25a.html %V 305 %X Ensuring the robustness of robotic systems is crucial for their deployment in safety-critical domains. Failure discovery, or falsification, is a widely used approach for evaluating robustness, with recent advancements focusing on improving sample efficiency and generalization through probabilistic sampling techniques and learning-theoretic approaches. However, existing methods typically rely on explicitly defined analytical cost functions to characterize failures, often overlooking the underlying causes and diversity of discovered failure scenarios. In this work, we propose a novel failure discovery framework that integrates contextual reasoning in the falsification process, specifically tailored for high evaluation-cost applications. Our method incorporates expert-in-the-loop feedback to construct a probabilistic surrogate model of failures using Bayesian inference. This model is iteratively refined and leveraged to guide an active learning strategy that prioritizes the discovery of diverse failure cases. We empirically validate our approach across a range of tasks for high-cost contextual falsification in robotic manipulation and autonomous driving.
APA
Parashar, A., Zhang, J., Li, Y. & Fan, C.. (2025). Cost-aware Discovery of Contextual Failures using Bayesian Active Learning. Proceedings of The 9th Conference on Robot Learning, in Proceedings of Machine Learning Research 305:2239-2267 Available from https://proceedings.mlr.press/v305/parashar25a.html.

Related Material