Robust Guarantees for Perception-Based Control

Sarah Dean, Nikolai Matni, Benjamin Recht, Vickie Ye
Proceedings of the 2nd Conference on Learning for Dynamics and Control, PMLR 120:350-360, 2020.

Abstract

Motivated by vision-based control of autonomous vehicles, we consider the problem of controlling a known linear dynamical system for which partial state information, such as vehicle position, is extracted from complex and nonlinear data, such as a camera image. Our approach is to use a learned perception map that predicts some linear function of the state and to design a corresponding safe set and robust controller for the closed loop system with this sensing scheme. We show that under suitable smoothness assumptions on both the perception map and the generative model relating state to complex and nonlinear data, parameters of the safe set can be learned via appropriately dense sampling of the state space. We then prove that the resulting perception-control loop has favorable generalization properties. We illustrate the usefulness of our approach on a synthetic example and on the self-driving car simulation platform CARLA.

Cite this Paper

BibTeX
@InProceedings{pmlr-v120-dean20a, title = {Robust Guarantees for Perception-Based Control}, author = {Dean, Sarah and Matni, Nikolai and Recht, Benjamin and Ye, Vickie}, booktitle = {Proceedings of the 2nd Conference on Learning for Dynamics and Control}, pages = {350--360}, year = {2020}, editor = {Bayen, Alexandre M. and Jadbabaie, Ali and Pappas, George and Parrilo, Pablo A. and Recht, Benjamin and Tomlin, Claire and Zeilinger, Melanie}, volume = {120}, series = {Proceedings of Machine Learning Research}, month = {10--11 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v120/dean20a/dean20a.pdf}, url = {https://proceedings.mlr.press/v120/dean20a.html}, abstract = {Motivated by vision-based control of autonomous vehicles, we consider the problem of controlling a known linear dynamical system for which partial state information, such as vehicle position, is extracted from complex and nonlinear data, such as a camera image. Our approach is to use a learned perception map that predicts some linear function of the state and to design a corresponding safe set and robust controller for the closed loop system with this sensing scheme. We show that under suitable smoothness assumptions on both the perception map and the generative model relating state to complex and nonlinear data, parameters of the safe set can be learned via appropriately dense sampling of the state space. We then prove that the resulting perception-control loop has favorable generalization properties. We illustrate the usefulness of our approach on a synthetic example and on the self-driving car simulation platform CARLA.} }
Endnote
%0 Conference Paper %T Robust Guarantees for Perception-Based Control %A Sarah Dean %A Nikolai Matni %A Benjamin Recht %A Vickie Ye %B Proceedings of the 2nd Conference on Learning for Dynamics and Control %C Proceedings of Machine Learning Research %D 2020 %E Alexandre M. Bayen %E Ali Jadbabaie %E George Pappas %E Pablo A. Parrilo %E Benjamin Recht %E Claire Tomlin %E Melanie Zeilinger %F pmlr-v120-dean20a %I PMLR %P 350--360 %U https://proceedings.mlr.press/v120/dean20a.html %V 120 %X Motivated by vision-based control of autonomous vehicles, we consider the problem of controlling a known linear dynamical system for which partial state information, such as vehicle position, is extracted from complex and nonlinear data, such as a camera image. Our approach is to use a learned perception map that predicts some linear function of the state and to design a corresponding safe set and robust controller for the closed loop system with this sensing scheme. We show that under suitable smoothness assumptions on both the perception map and the generative model relating state to complex and nonlinear data, parameters of the safe set can be learned via appropriately dense sampling of the state space. We then prove that the resulting perception-control loop has favorable generalization properties. We illustrate the usefulness of our approach on a synthetic example and on the self-driving car simulation platform CARLA.
APA
Dean, S., Matni, N., Recht, B. & Ye, V.. (2020). Robust Guarantees for Perception-Based Control. Proceedings of the 2nd Conference on Learning for Dynamics and Control, in Proceedings of Machine Learning Research 120:350-360 Available from https://proceedings.mlr.press/v120/dean20a.html.

Related Material