Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula

Eli Bronstein, Sirish Srinivasan, Supratik Paul, Aman Sinha, Matthew O’Kelly, Payam Nikdel, Shimon Whiteson
Proceedings of The 6th Conference on Robot Learning, PMLR 205:188-198, 2023.

Abstract

ML-based motion planning is a promising approach to produce agents that exhibit complex behaviors, and automatically adapt to novel environments. In the context of autonomous driving, it is common to treat all available training data equally. However, this approach produces agents that do not perform robustly in safety-critical settings, an issue that cannot be addressed by simply adding more data to the training set – we show that an agent trained using only a 10% subset of the data performs just as well as an agent trained on the entire dataset. We present a method to predict the inherent difficulty of a driving situation given data collected from a fleet of autonomous vehicles deployed on public roads. We then demonstrate that this difficulty score can be used in a zero-shot transfer to generate curricula for an imitation-learning based planning agent. Compared to training on the entire unbiased training dataset, we show that prioritizing difficult driving scenarios both reduces collisions by 15% and increases route adherence by 14% in closed-loop evaluation, all while using only 10% of the training data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-bronstein23a, title = {Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula}, author = {Bronstein, Eli and Srinivasan, Sirish and Paul, Supratik and Sinha, Aman and O'Kelly, Matthew and Nikdel, Payam and Whiteson, Shimon}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {188--198}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/bronstein23a/bronstein23a.pdf}, url = {https://proceedings.mlr.press/v205/bronstein23a.html}, abstract = {ML-based motion planning is a promising approach to produce agents that exhibit complex behaviors, and automatically adapt to novel environments. In the context of autonomous driving, it is common to treat all available training data equally. However, this approach produces agents that do not perform robustly in safety-critical settings, an issue that cannot be addressed by simply adding more data to the training set – we show that an agent trained using only a 10% subset of the data performs just as well as an agent trained on the entire dataset. We present a method to predict the inherent difficulty of a driving situation given data collected from a fleet of autonomous vehicles deployed on public roads. We then demonstrate that this difficulty score can be used in a zero-shot transfer to generate curricula for an imitation-learning based planning agent. Compared to training on the entire unbiased training dataset, we show that prioritizing difficult driving scenarios both reduces collisions by 15% and increases route adherence by 14% in closed-loop evaluation, all while using only 10% of the training data.} }
Endnote
%0 Conference Paper %T Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula %A Eli Bronstein %A Sirish Srinivasan %A Supratik Paul %A Aman Sinha %A Matthew O’Kelly %A Payam Nikdel %A Shimon Whiteson %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-bronstein23a %I PMLR %P 188--198 %U https://proceedings.mlr.press/v205/bronstein23a.html %V 205 %X ML-based motion planning is a promising approach to produce agents that exhibit complex behaviors, and automatically adapt to novel environments. In the context of autonomous driving, it is common to treat all available training data equally. However, this approach produces agents that do not perform robustly in safety-critical settings, an issue that cannot be addressed by simply adding more data to the training set – we show that an agent trained using only a 10% subset of the data performs just as well as an agent trained on the entire dataset. We present a method to predict the inherent difficulty of a driving situation given data collected from a fleet of autonomous vehicles deployed on public roads. We then demonstrate that this difficulty score can be used in a zero-shot transfer to generate curricula for an imitation-learning based planning agent. Compared to training on the entire unbiased training dataset, we show that prioritizing difficult driving scenarios both reduces collisions by 15% and increases route adherence by 14% in closed-loop evaluation, all while using only 10% of the training data.
APA
Bronstein, E., Srinivasan, S., Paul, S., Sinha, A., O’Kelly, M., Nikdel, P. & Whiteson, S.. (2023). Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:188-198 Available from https://proceedings.mlr.press/v205/bronstein23a.html.

Related Material