Grounding Robot Plans from Natural Language Instructions with Incomplete World Knowledge

Daniel Nyga, Subhro Roy, Rohan Paul, Daehyung Park, Mihai Pomarlan, Michael Beetz, Nicholas Roy
Proceedings of The 2nd Conference on Robot Learning, PMLR 87:714-723, 2018.

Abstract

Our goal is to enable robots to interpret and execute high-level tasks conveyed using natural language instructions. For example, consider tasking a household robot to, “prepare my breakfast”, “clear the boxes on the table” or “make me a fruit milkshake”. Interpreting such underspecified instructions requires environmental context and background knowledge about how to accomplish complex tasks. Further, the robot’s workspace knowledge may be incomplete: the environment may only be partially-observed or background knowledge may be missing causing a failure in plan synthesis. We introduce a probabilistic model that utilizes background knowledge to infer latent or missing plan constituents based on semantic co-associations learned from noisy textual corpora of task descriptions. The ability to infer missing plan constituents enables information-seeking actions such as visual exploration or dialogue with the human to acquire new knowledge to fill incomplete plans. Results indicate robust plan inference from under-specified instructions in partially-known worlds.

Cite this Paper

BibTeX
@InProceedings{pmlr-v87-nyga18a, title = {Grounding Robot Plans from Natural Language Instructions with Incomplete World Knowledge}, author = {Nyga, Daniel and Roy, Subhro and Paul, Rohan and Park, Daehyung and Pomarlan, Mihai and Beetz, Michael and Roy, Nicholas}, booktitle = {Proceedings of The 2nd Conference on Robot Learning}, pages = {714--723}, year = {2018}, editor = {Billard, Aude and Dragan, Anca and Peters, Jan and Morimoto, Jun}, volume = {87}, series = {Proceedings of Machine Learning Research}, month = {29--31 Oct}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v87/nyga18a/nyga18a.pdf}, url = {https://proceedings.mlr.press/v87/nyga18a.html}, abstract = {Our goal is to enable robots to interpret and execute high-level tasks conveyed using natural language instructions. For example, consider tasking a household robot to, “prepare my breakfast”, “clear the boxes on the table” or “make me a fruit milkshake”. Interpreting such underspecified instructions requires environmental context and background knowledge about how to accomplish complex tasks. Further, the robot’s workspace knowledge may be incomplete: the environment may only be partially-observed or background knowledge may be missing causing a failure in plan synthesis. We introduce a probabilistic model that utilizes background knowledge to infer latent or missing plan constituents based on semantic co-associations learned from noisy textual corpora of task descriptions. The ability to infer missing plan constituents enables information-seeking actions such as visual exploration or dialogue with the human to acquire new knowledge to fill incomplete plans. Results indicate robust plan inference from under-specified instructions in partially-known worlds. } }
Endnote
%0 Conference Paper %T Grounding Robot Plans from Natural Language Instructions with Incomplete World Knowledge %A Daniel Nyga %A Subhro Roy %A Rohan Paul %A Daehyung Park %A Mihai Pomarlan %A Michael Beetz %A Nicholas Roy %B Proceedings of The 2nd Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2018 %E Aude Billard %E Anca Dragan %E Jan Peters %E Jun Morimoto %F pmlr-v87-nyga18a %I PMLR %P 714--723 %U https://proceedings.mlr.press/v87/nyga18a.html %V 87 %X Our goal is to enable robots to interpret and execute high-level tasks conveyed using natural language instructions. For example, consider tasking a household robot to, “prepare my breakfast”, “clear the boxes on the table” or “make me a fruit milkshake”. Interpreting such underspecified instructions requires environmental context and background knowledge about how to accomplish complex tasks. Further, the robot’s workspace knowledge may be incomplete: the environment may only be partially-observed or background knowledge may be missing causing a failure in plan synthesis. We introduce a probabilistic model that utilizes background knowledge to infer latent or missing plan constituents based on semantic co-associations learned from noisy textual corpora of task descriptions. The ability to infer missing plan constituents enables information-seeking actions such as visual exploration or dialogue with the human to acquire new knowledge to fill incomplete plans. Results indicate robust plan inference from under-specified instructions in partially-known worlds.
APA
Nyga, D., Roy, S., Paul, R., Park, D., Pomarlan, M., Beetz, M. & Roy, N.. (2018). Grounding Robot Plans from Natural Language Instructions with Incomplete World Knowledge. Proceedings of The 2nd Conference on Robot Learning, in Proceedings of Machine Learning Research 87:714-723 Available from https://proceedings.mlr.press/v87/nyga18a.html.

Related Material