Learning Stability Attention in Vision-based End-to-end Driving Policies

Tsun-Hsuan Wang, Wei Xiao, Makram Chahine, Alexander Amini, Ramin Hasani, Daniela Rus
Proceedings of The 5th Annual Learning for Dynamics and Control Conference, PMLR 211:1099-1111, 2023.

Abstract

Today’s end-to-end learning systems can learn to explicitly infer control from perception. However, it is difficult to guarantee stability and robustness for these systems since they are often exposed to unstructured, high-dimensional, and complex observation spaces (e.g., autonomous driving from a stream of pixel inputs). We propose to leverage control Lyapunov functions (CLFs) to equip end-to-end vision-based policies with stability properties and introduce stability attention in CLFs (att-CLFs) to tackle environmental changes and improve learning flexibility. We also present an uncertainty propagation technique that is tightly integrated into att-CLFs. We demonstrate the effectiveness of att-CLFs via comparison with classical CLFs, model predictive control, and vanilla end-to-end learning in a photo-realistic simulator and on a real full-scale autonomous vehicle.

Cite this Paper

BibTeX
@InProceedings{pmlr-v211-wang23b, title = {Learning Stability Attention in Vision-based End-to-end Driving Policies}, author = {Wang, Tsun-Hsuan and Xiao, Wei and Chahine, Makram and Amini, Alexander and Hasani, Ramin and Rus, Daniela}, booktitle = {Proceedings of The 5th Annual Learning for Dynamics and Control Conference}, pages = {1099--1111}, year = {2023}, editor = {Matni, Nikolai and Morari, Manfred and Pappas, George J.}, volume = {211}, series = {Proceedings of Machine Learning Research}, month = {15--16 Jun}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v211/wang23b/wang23b.pdf}, url = {https://proceedings.mlr.press/v211/wang23b.html}, abstract = {Today’s end-to-end learning systems can learn to explicitly infer control from perception. However, it is difficult to guarantee stability and robustness for these systems since they are often exposed to unstructured, high-dimensional, and complex observation spaces (e.g., autonomous driving from a stream of pixel inputs). We propose to leverage control Lyapunov functions (CLFs) to equip end-to-end vision-based policies with stability properties and introduce stability attention in CLFs (att-CLFs) to tackle environmental changes and improve learning flexibility. We also present an uncertainty propagation technique that is tightly integrated into att-CLFs. We demonstrate the effectiveness of att-CLFs via comparison with classical CLFs, model predictive control, and vanilla end-to-end learning in a photo-realistic simulator and on a real full-scale autonomous vehicle.} }
Endnote
%0 Conference Paper %T Learning Stability Attention in Vision-based End-to-end Driving Policies %A Tsun-Hsuan Wang %A Wei Xiao %A Makram Chahine %A Alexander Amini %A Ramin Hasani %A Daniela Rus %B Proceedings of The 5th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2023 %E Nikolai Matni %E Manfred Morari %E George J. Pappas %F pmlr-v211-wang23b %I PMLR %P 1099--1111 %U https://proceedings.mlr.press/v211/wang23b.html %V 211 %X Today’s end-to-end learning systems can learn to explicitly infer control from perception. However, it is difficult to guarantee stability and robustness for these systems since they are often exposed to unstructured, high-dimensional, and complex observation spaces (e.g., autonomous driving from a stream of pixel inputs). We propose to leverage control Lyapunov functions (CLFs) to equip end-to-end vision-based policies with stability properties and introduce stability attention in CLFs (att-CLFs) to tackle environmental changes and improve learning flexibility. We also present an uncertainty propagation technique that is tightly integrated into att-CLFs. We demonstrate the effectiveness of att-CLFs via comparison with classical CLFs, model predictive control, and vanilla end-to-end learning in a photo-realistic simulator and on a real full-scale autonomous vehicle.
APA
Wang, T., Xiao, W., Chahine, M., Amini, A., Hasani, R. & Rus, D.. (2023). Learning Stability Attention in Vision-based End-to-end Driving Policies. Proceedings of The 5th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 211:1099-1111 Available from https://proceedings.mlr.press/v211/wang23b.html.

Related Material