Rate-Informed Discovery via Bayesian Adaptive Multifidelity Sampling

Aman Sinha, Payam Nikdel, Supratik Paul, Shimon Whiteson
Proceedings of The 8th Conference on Robot Learning, PMLR 270:2579-2598, 2025.

Abstract

Ensuring the safety of autonomous vehicles (AVs) requires both accurate estimation of their performance and efficient discovery of potential failure cases. This paper introduces Bayesian adaptive multifidelity sampling (BAMS), which leverages the power of adaptive Bayesian sampling to achieve efficient discovery while simultaneously estimating the rate of adverse events. BAMS prioritizes exploration of regions with potentially low performance, leading to the identification of novel and critical scenarios that traditional methods might miss. Using real-world AV data we demonstrate that BAMS discovers 10 times as many issues as Monte Carlo (MC) and importance sampling (IS) baselines, while at the same time generating rate estimates with variances 15 and 6 times narrower than MC and IS baselines respectively.

Cite this Paper

BibTeX
@InProceedings{pmlr-v270-sinha25a, title = {Rate-Informed Discovery via Bayesian Adaptive Multifidelity Sampling}, author = {Sinha, Aman and Nikdel, Payam and Paul, Supratik and Whiteson, Shimon}, booktitle = {Proceedings of The 8th Conference on Robot Learning}, pages = {2579--2598}, year = {2025}, editor = {Agrawal, Pulkit and Kroemer, Oliver and Burgard, Wolfram}, volume = {270}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v270/main/assets/sinha25a/sinha25a.pdf}, url = {https://proceedings.mlr.press/v270/sinha25a.html}, abstract = {Ensuring the safety of autonomous vehicles (AVs) requires both accurate estimation of their performance and efficient discovery of potential failure cases. This paper introduces Bayesian adaptive multifidelity sampling (BAMS), which leverages the power of adaptive Bayesian sampling to achieve efficient discovery while simultaneously estimating the rate of adverse events. BAMS prioritizes exploration of regions with potentially low performance, leading to the identification of novel and critical scenarios that traditional methods might miss. Using real-world AV data we demonstrate that BAMS discovers 10 times as many issues as Monte Carlo (MC) and importance sampling (IS) baselines, while at the same time generating rate estimates with variances 15 and 6 times narrower than MC and IS baselines respectively.} }
Endnote
%0 Conference Paper %T Rate-Informed Discovery via Bayesian Adaptive Multifidelity Sampling %A Aman Sinha %A Payam Nikdel %A Supratik Paul %A Shimon Whiteson %B Proceedings of The 8th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Pulkit Agrawal %E Oliver Kroemer %E Wolfram Burgard %F pmlr-v270-sinha25a %I PMLR %P 2579--2598 %U https://proceedings.mlr.press/v270/sinha25a.html %V 270 %X Ensuring the safety of autonomous vehicles (AVs) requires both accurate estimation of their performance and efficient discovery of potential failure cases. This paper introduces Bayesian adaptive multifidelity sampling (BAMS), which leverages the power of adaptive Bayesian sampling to achieve efficient discovery while simultaneously estimating the rate of adverse events. BAMS prioritizes exploration of regions with potentially low performance, leading to the identification of novel and critical scenarios that traditional methods might miss. Using real-world AV data we demonstrate that BAMS discovers 10 times as many issues as Monte Carlo (MC) and importance sampling (IS) baselines, while at the same time generating rate estimates with variances 15 and 6 times narrower than MC and IS baselines respectively.
APA
Sinha, A., Nikdel, P., Paul, S. & Whiteson, S.. (2025). Rate-Informed Discovery via Bayesian Adaptive Multifidelity Sampling. Proceedings of The 8th Conference on Robot Learning, in Proceedings of Machine Learning Research 270:2579-2598 Available from https://proceedings.mlr.press/v270/sinha25a.html.

Related Material