A Framework for Learning to Request Rich and Contextually Useful Information from Humans

Khanh X Nguyen, Yonatan Bisk, Hal Daumé Iii
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:16553-16568, 2022.

Abstract

When deployed, AI agents will encounter problems that are beyond their autonomous problem-solving capabilities. Leveraging human assistance can help agents overcome their inherent limitations and robustly cope with unfamiliar situations. We present a general interactive framework that enables an agent to request and interpret rich, contextually useful information from an assistant that has knowledge about the task and the environment. We demonstrate the practicality of our framework on a simulated human-assisted navigation problem. Aided with an assistance-requesting policy learned by our method, a navigation agent achieves up to a 7{\texttimes} improvement in success rate on tasks that take place in previously unseen environments, compared to fully autonomous behavior. We show that the agent can take advantage of different types of information depending on the context, and analyze the benefits and challenges of learning the assistance-requesting policy when the assistant can recursively decompose tasks into subtasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-nguyen22a, title = {A Framework for Learning to Request Rich and Contextually Useful Information from Humans}, author = {Nguyen, Khanh X and Bisk, Yonatan and Iii, Hal Daum{\'e}}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {16553--16568}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/nguyen22a/nguyen22a.pdf}, url = {https://proceedings.mlr.press/v162/nguyen22a.html}, abstract = {When deployed, AI agents will encounter problems that are beyond their autonomous problem-solving capabilities. Leveraging human assistance can help agents overcome their inherent limitations and robustly cope with unfamiliar situations. We present a general interactive framework that enables an agent to request and interpret rich, contextually useful information from an assistant that has knowledge about the task and the environment. We demonstrate the practicality of our framework on a simulated human-assisted navigation problem. Aided with an assistance-requesting policy learned by our method, a navigation agent achieves up to a 7{\texttimes} improvement in success rate on tasks that take place in previously unseen environments, compared to fully autonomous behavior. We show that the agent can take advantage of different types of information depending on the context, and analyze the benefits and challenges of learning the assistance-requesting policy when the assistant can recursively decompose tasks into subtasks.} }
Endnote
%0 Conference Paper %T A Framework for Learning to Request Rich and Contextually Useful Information from Humans %A Khanh X Nguyen %A Yonatan Bisk %A Hal Daumé Iii %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-nguyen22a %I PMLR %P 16553--16568 %U https://proceedings.mlr.press/v162/nguyen22a.html %V 162 %X When deployed, AI agents will encounter problems that are beyond their autonomous problem-solving capabilities. Leveraging human assistance can help agents overcome their inherent limitations and robustly cope with unfamiliar situations. We present a general interactive framework that enables an agent to request and interpret rich, contextually useful information from an assistant that has knowledge about the task and the environment. We demonstrate the practicality of our framework on a simulated human-assisted navigation problem. Aided with an assistance-requesting policy learned by our method, a navigation agent achieves up to a 7{\texttimes} improvement in success rate on tasks that take place in previously unseen environments, compared to fully autonomous behavior. We show that the agent can take advantage of different types of information depending on the context, and analyze the benefits and challenges of learning the assistance-requesting policy when the assistant can recursively decompose tasks into subtasks.
APA
Nguyen, K.X., Bisk, Y. & Iii, H.D.. (2022). A Framework for Learning to Request Rich and Contextually Useful Information from Humans. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:16553-16568 Available from https://proceedings.mlr.press/v162/nguyen22a.html.

Related Material