Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction

Valts Blukis, Dipendra Misra, Ross A. Knepper, Yoav Artzi
Proceedings of The 2nd Conference on Robot Learning, PMLR 87:505-518, 2018.

Abstract

We propose an approach for mapping natural language instructions and raw observations to continuous control of a quadcopter drone. Our model predicts interpretable position-visitation distributions indicating where the agent should go during execution and where it should stop, and uses the predicted distributions to select the actions to execute. This two-step model decomposition allows for simple and efficient training using a combination of supervised learning and imitation learning. We evaluate our approach with a realistic drone simulator, and demonstrate absolute task-completion accuracy improvements of 16.85% over two state-of-the-art instruction-following methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v87-blukis18a, title = {Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction}, author = {Blukis, Valts and Misra, Dipendra and Knepper, Ross A. and Artzi, Yoav}, booktitle = {Proceedings of The 2nd Conference on Robot Learning}, pages = {505--518}, year = {2018}, editor = {Billard, Aude and Dragan, Anca and Peters, Jan and Morimoto, Jun}, volume = {87}, series = {Proceedings of Machine Learning Research}, month = {29--31 Oct}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v87/blukis18a/blukis18a.pdf}, url = {https://proceedings.mlr.press/v87/blukis18a.html}, abstract = {We propose an approach for mapping natural language instructions and raw observations to continuous control of a quadcopter drone. Our model predicts interpretable position-visitation distributions indicating where the agent should go during execution and where it should stop, and uses the predicted distributions to select the actions to execute. This two-step model decomposition allows for simple and efficient training using a combination of supervised learning and imitation learning. We evaluate our approach with a realistic drone simulator, and demonstrate absolute task-completion accuracy improvements of 16.85% over two state-of-the-art instruction-following methods. } }
Endnote
%0 Conference Paper %T Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction %A Valts Blukis %A Dipendra Misra %A Ross A. Knepper %A Yoav Artzi %B Proceedings of The 2nd Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2018 %E Aude Billard %E Anca Dragan %E Jan Peters %E Jun Morimoto %F pmlr-v87-blukis18a %I PMLR %P 505--518 %U https://proceedings.mlr.press/v87/blukis18a.html %V 87 %X We propose an approach for mapping natural language instructions and raw observations to continuous control of a quadcopter drone. Our model predicts interpretable position-visitation distributions indicating where the agent should go during execution and where it should stop, and uses the predicted distributions to select the actions to execute. This two-step model decomposition allows for simple and efficient training using a combination of supervised learning and imitation learning. We evaluate our approach with a realistic drone simulator, and demonstrate absolute task-completion accuracy improvements of 16.85% over two state-of-the-art instruction-following methods.
APA
Blukis, V., Misra, D., Knepper, R.A. & Artzi, Y.. (2018). Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction. Proceedings of The 2nd Conference on Robot Learning, in Proceedings of Machine Learning Research 87:505-518 Available from https://proceedings.mlr.press/v87/blukis18a.html.

Related Material