Grounding Robot Plans from Natural Language Instructions with Incomplete World Knowledge

Daniel Nyga, Subhro Roy, Rohan Paul, Daehyung Park, Mihai Pomarlan, Michael Beetz, Nicholas Roy
; Proceedings of The 2nd Conference on Robot Learning, PMLR 87:714-723, 2018.

Abstract

Our goal is to enable robots to interpret and execute high-level tasks conveyed using natural language instructions. For example, consider tasking a household robot to, “prepare my breakfast”, “clear the boxes on the table” or “make me a fruit milkshake”. Interpreting such underspecified instructions requires environmental context and background knowledge about how to accomplish complex tasks. Further, the robot’s workspace knowledge may be incomplete: the environment may only be partially-observed or background knowledge may be missing causing a failure in plan synthesis. We introduce a probabilistic model that utilizes background knowledge to infer latent or missing plan constituents based on semantic co-associations learned from noisy textual corpora of task descriptions. The ability to infer missing plan constituents enables information-seeking actions such as visual exploration or dialogue with the human to acquire new knowledge to fill incomplete plans. Results indicate robust plan inference from under-specified instructions in partially-known worlds.

Cite this Paper


BibTeX
@InProceedings{pmlr-v87-nyga18a, title = {Grounding Robot Plans from Natural Language Instructions with Incomplete World Knowledge}, author = {Nyga, Daniel and Roy, Subhro and Paul, Rohan and Park, Daehyung and Pomarlan, Mihai and Beetz, Michael and Roy, Nicholas}, booktitle = {Proceedings of The 2nd Conference on Robot Learning}, pages = {714--723}, year = {2018}, editor = {Aude Billard and Anca Dragan and Jan Peters and Jun Morimoto}, volume = {87}, series = {Proceedings of Machine Learning Research}, address = {}, month = {29--31 Oct}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v87/nyga18a/nyga18a.pdf}, url = {http://proceedings.mlr.press/v87/nyga18a.html}, abstract = {Our goal is to enable robots to interpret and execute high-level tasks conveyed using natural language instructions. For example, consider tasking a household robot to, “prepare my breakfast”, “clear the boxes on the table” or “make me a fruit milkshake”. Interpreting such underspecified instructions requires environmental context and background knowledge about how to accomplish complex tasks. Further, the robot’s workspace knowledge may be incomplete: the environment may only be partially-observed or background knowledge may be missing causing a failure in plan synthesis. We introduce a probabilistic model that utilizes background knowledge to infer latent or missing plan constituents based on semantic co-associations learned from noisy textual corpora of task descriptions. The ability to infer missing plan constituents enables information-seeking actions such as visual exploration or dialogue with the human to acquire new knowledge to fill incomplete plans. Results indicate robust plan inference from under-specified instructions in partially-known worlds. } }
Endnote
%0 Conference Paper %T Grounding Robot Plans from Natural Language Instructions with Incomplete World Knowledge %A Daniel Nyga %A Subhro Roy %A Rohan Paul %A Daehyung Park %A Mihai Pomarlan %A Michael Beetz %A Nicholas Roy %B Proceedings of The 2nd Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2018 %E Aude Billard %E Anca Dragan %E Jan Peters %E Jun Morimoto %F pmlr-v87-nyga18a %I PMLR %J Proceedings of Machine Learning Research %P 714--723 %U http://proceedings.mlr.press %V 87 %W PMLR %X Our goal is to enable robots to interpret and execute high-level tasks conveyed using natural language instructions. For example, consider tasking a household robot to, “prepare my breakfast”, “clear the boxes on the table” or “make me a fruit milkshake”. Interpreting such underspecified instructions requires environmental context and background knowledge about how to accomplish complex tasks. Further, the robot’s workspace knowledge may be incomplete: the environment may only be partially-observed or background knowledge may be missing causing a failure in plan synthesis. We introduce a probabilistic model that utilizes background knowledge to infer latent or missing plan constituents based on semantic co-associations learned from noisy textual corpora of task descriptions. The ability to infer missing plan constituents enables information-seeking actions such as visual exploration or dialogue with the human to acquire new knowledge to fill incomplete plans. Results indicate robust plan inference from under-specified instructions in partially-known worlds.
APA
Nyga, D., Roy, S., Paul, R., Park, D., Pomarlan, M., Beetz, M. & Roy, N.. (2018). Grounding Robot Plans from Natural Language Instructions with Incomplete World Knowledge. Proceedings of The 2nd Conference on Robot Learning, in PMLR 87:714-723

Related Material