Differentiable Logic Layer for Rule Guided Trajectory Prediction

Xiao Li, Guy Rosman, Igor Gilitschenski, Jonathan DeCastro, Cristian-Ioan Vasile, Sertac Karaman, Daniela Rus
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:2178-2194, 2021.

Abstract

In this work, we propose a method for integration of temporal logic formulas into a neural network. Our main contribution is a new logic optimization layer that uses differentiable optimization on the formulas’ robustness function. This allows incorporating traffic rules into deep learning based trajectory prediction approaches. In the forward pass, an initial prediction from a base predictor is used to initialize and guide the robustness optimization process. Backpropagation through the logic layer allows for simultaneously adjusting the parameters of the rules and the initial prediction network. The integration of a logic layer affords both improved predictions, as well as quantification rule satisfaction and violation during predictor execution. As such, it can serve as a parametric safety- envelope for black box behavior models. We demonstrate how integrating traffic rules improves the predictor performance using real traffic data from the NuScenes dataset.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-li21b, title = {Differentiable Logic Layer for Rule Guided Trajectory Prediction}, author = {Li, Xiao and Rosman, Guy and Gilitschenski, Igor and DeCastro, Jonathan and Vasile, Cristian-Ioan and Karaman, Sertac and Rus, Daniela}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {2178--2194}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/li21b/li21b.pdf}, url = {https://proceedings.mlr.press/v155/li21b.html}, abstract = {In this work, we propose a method for integration of temporal logic formulas into a neural network. Our main contribution is a new logic optimization layer that uses differentiable optimization on the formulas’ robustness function. This allows incorporating traffic rules into deep learning based trajectory prediction approaches. In the forward pass, an initial prediction from a base predictor is used to initialize and guide the robustness optimization process. Backpropagation through the logic layer allows for simultaneously adjusting the parameters of the rules and the initial prediction network. The integration of a logic layer affords both improved predictions, as well as quantification rule satisfaction and violation during predictor execution. As such, it can serve as a parametric safety- envelope for black box behavior models. We demonstrate how integrating traffic rules improves the predictor performance using real traffic data from the NuScenes dataset.} }
Endnote
%0 Conference Paper %T Differentiable Logic Layer for Rule Guided Trajectory Prediction %A Xiao Li %A Guy Rosman %A Igor Gilitschenski %A Jonathan DeCastro %A Cristian-Ioan Vasile %A Sertac Karaman %A Daniela Rus %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-li21b %I PMLR %P 2178--2194 %U https://proceedings.mlr.press/v155/li21b.html %V 155 %X In this work, we propose a method for integration of temporal logic formulas into a neural network. Our main contribution is a new logic optimization layer that uses differentiable optimization on the formulas’ robustness function. This allows incorporating traffic rules into deep learning based trajectory prediction approaches. In the forward pass, an initial prediction from a base predictor is used to initialize and guide the robustness optimization process. Backpropagation through the logic layer allows for simultaneously adjusting the parameters of the rules and the initial prediction network. The integration of a logic layer affords both improved predictions, as well as quantification rule satisfaction and violation during predictor execution. As such, it can serve as a parametric safety- envelope for black box behavior models. We demonstrate how integrating traffic rules improves the predictor performance using real traffic data from the NuScenes dataset.
APA
Li, X., Rosman, G., Gilitschenski, I., DeCastro, J., Vasile, C., Karaman, S. & Rus, D.. (2021). Differentiable Logic Layer for Rule Guided Trajectory Prediction. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:2178-2194 Available from https://proceedings.mlr.press/v155/li21b.html.

Related Material