Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments

Jason Xinyu Liu, Ziyi Yang, Ifrah Idrees, Sam Liang, Benjamin Schornstein, Stefanie Tellex, Ankit Shah
Proceedings of The 7th Conference on Robot Learning, PMLR 229:1084-1110, 2023.

Abstract

Grounding navigational commands to linear temporal logic (LTL) leverages its unambiguous semantics for reasoning about long-horizon tasks and verifying the satisfaction of temporal constraints. Existing approaches require training data from the specific environment and landmarks that will be used in natural language to understand commands in those environments. We propose Lang2LTL, a modular system and a software package that leverages large language models (LLMs) to ground temporal navigational commands to LTL specifications in environments without prior language data. We comprehensively evaluate Lang2LTL for five well-defined generalization behaviors. Lang2LTL demonstrates the state-of-the-art ability of a single model to ground navigational commands to diverse temporal specifications in 21 city-scaled environments. Finally, we demonstrate a physical robot using Lang2LTL can follow 52 semantically diverse navigational commands in two indoor environments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-liu23d, title = {Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments}, author = {Liu, Jason Xinyu and Yang, Ziyi and Idrees, Ifrah and Liang, Sam and Schornstein, Benjamin and Tellex, Stefanie and Shah, Ankit}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {1084--1110}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/liu23d/liu23d.pdf}, url = {https://proceedings.mlr.press/v229/liu23d.html}, abstract = {Grounding navigational commands to linear temporal logic (LTL) leverages its unambiguous semantics for reasoning about long-horizon tasks and verifying the satisfaction of temporal constraints. Existing approaches require training data from the specific environment and landmarks that will be used in natural language to understand commands in those environments. We propose Lang2LTL, a modular system and a software package that leverages large language models (LLMs) to ground temporal navigational commands to LTL specifications in environments without prior language data. We comprehensively evaluate Lang2LTL for five well-defined generalization behaviors. Lang2LTL demonstrates the state-of-the-art ability of a single model to ground navigational commands to diverse temporal specifications in 21 city-scaled environments. Finally, we demonstrate a physical robot using Lang2LTL can follow 52 semantically diverse navigational commands in two indoor environments.} }
Endnote
%0 Conference Paper %T Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments %A Jason Xinyu Liu %A Ziyi Yang %A Ifrah Idrees %A Sam Liang %A Benjamin Schornstein %A Stefanie Tellex %A Ankit Shah %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-liu23d %I PMLR %P 1084--1110 %U https://proceedings.mlr.press/v229/liu23d.html %V 229 %X Grounding navigational commands to linear temporal logic (LTL) leverages its unambiguous semantics for reasoning about long-horizon tasks and verifying the satisfaction of temporal constraints. Existing approaches require training data from the specific environment and landmarks that will be used in natural language to understand commands in those environments. We propose Lang2LTL, a modular system and a software package that leverages large language models (LLMs) to ground temporal navigational commands to LTL specifications in environments without prior language data. We comprehensively evaluate Lang2LTL for five well-defined generalization behaviors. Lang2LTL demonstrates the state-of-the-art ability of a single model to ground navigational commands to diverse temporal specifications in 21 city-scaled environments. Finally, we demonstrate a physical robot using Lang2LTL can follow 52 semantically diverse navigational commands in two indoor environments.
APA
Liu, J.X., Yang, Z., Idrees, I., Liang, S., Schornstein, B., Tellex, S. & Shah, A.. (2023). Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:1084-1110 Available from https://proceedings.mlr.press/v229/liu23d.html.

Related Material