Modular Vehicle Control for Transferring Semantic Information Between Weather Conditions Using GANs

Patrick Wenzel, Qadeer Khan, Daniel Cremers, Laura Leal-Taixe
Proceedings of The 2nd Conference on Robot Learning, PMLR 87:253-269, 2018.

Abstract

Even though end-to-end supervised learning has shown promising results for sensorimotor control of self-driving cars, its performance is greatly affected by the weather conditions under which it was trained, showing poor generalization to unseen conditions. In this paper, we show how knowledge can be transferred using semantic maps to new weather conditions without the need to obtain new ground truth data. To this end, we propose to divide the task of vehicle control into two independent modules: a control module which is only trained on one weather condition for which labeled steering data is available, and a perception module which is used as an interface between new weather conditions and the fixed control module. To generate the semantic data needed to train the perception module, we propose to use a generative adversarial network (GAN)-based model to retrieve the semantic information for the new conditions in an unsupervised manner. We introduce a master-servant architecture, where the master model (semantic labels available) trains the servant model (semantic labels not available). We show that our proposed method trained with ground truth data for a single weather condition is capable of achieving similar results on the task of steering angle prediction as an end-to-end model trained with ground truth data of 15 different weather conditions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v87-wenzel18a, title = {Modular Vehicle Control for Transferring Semantic Information Between Weather Conditions Using GANs}, author = {Wenzel, Patrick and Khan, Qadeer and Cremers, Daniel and Leal-Taixe, Laura}, booktitle = {Proceedings of The 2nd Conference on Robot Learning}, pages = {253--269}, year = {2018}, editor = {Billard, Aude and Dragan, Anca and Peters, Jan and Morimoto, Jun}, volume = {87}, series = {Proceedings of Machine Learning Research}, month = {29--31 Oct}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v87/wenzel18a/wenzel18a.pdf}, url = {https://proceedings.mlr.press/v87/wenzel18a.html}, abstract = {Even though end-to-end supervised learning has shown promising results for sensorimotor control of self-driving cars, its performance is greatly affected by the weather conditions under which it was trained, showing poor generalization to unseen conditions. In this paper, we show how knowledge can be transferred using semantic maps to new weather conditions without the need to obtain new ground truth data. To this end, we propose to divide the task of vehicle control into two independent modules: a control module which is only trained on one weather condition for which labeled steering data is available, and a perception module which is used as an interface between new weather conditions and the fixed control module. To generate the semantic data needed to train the perception module, we propose to use a generative adversarial network (GAN)-based model to retrieve the semantic information for the new conditions in an unsupervised manner. We introduce a master-servant architecture, where the master model (semantic labels available) trains the servant model (semantic labels not available). We show that our proposed method trained with ground truth data for a single weather condition is capable of achieving similar results on the task of steering angle prediction as an end-to-end model trained with ground truth data of 15 different weather conditions. } }
Endnote
%0 Conference Paper %T Modular Vehicle Control for Transferring Semantic Information Between Weather Conditions Using GANs %A Patrick Wenzel %A Qadeer Khan %A Daniel Cremers %A Laura Leal-Taixe %B Proceedings of The 2nd Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2018 %E Aude Billard %E Anca Dragan %E Jan Peters %E Jun Morimoto %F pmlr-v87-wenzel18a %I PMLR %P 253--269 %U https://proceedings.mlr.press/v87/wenzel18a.html %V 87 %X Even though end-to-end supervised learning has shown promising results for sensorimotor control of self-driving cars, its performance is greatly affected by the weather conditions under which it was trained, showing poor generalization to unseen conditions. In this paper, we show how knowledge can be transferred using semantic maps to new weather conditions without the need to obtain new ground truth data. To this end, we propose to divide the task of vehicle control into two independent modules: a control module which is only trained on one weather condition for which labeled steering data is available, and a perception module which is used as an interface between new weather conditions and the fixed control module. To generate the semantic data needed to train the perception module, we propose to use a generative adversarial network (GAN)-based model to retrieve the semantic information for the new conditions in an unsupervised manner. We introduce a master-servant architecture, where the master model (semantic labels available) trains the servant model (semantic labels not available). We show that our proposed method trained with ground truth data for a single weather condition is capable of achieving similar results on the task of steering angle prediction as an end-to-end model trained with ground truth data of 15 different weather conditions.
APA
Wenzel, P., Khan, Q., Cremers, D. & Leal-Taixe, L.. (2018). Modular Vehicle Control for Transferring Semantic Information Between Weather Conditions Using GANs. Proceedings of The 2nd Conference on Robot Learning, in Proceedings of Machine Learning Research 87:253-269 Available from https://proceedings.mlr.press/v87/wenzel18a.html.

Related Material