Understanding Teacher Gaze Patterns for Robot Learning

Akanksha Saran, Elaine Schaertl Short, Andrea Thomaz, Scott Niekum
Proceedings of the Conference on Robot Learning, PMLR 100:1247-1258, 2020.

Abstract

Human gaze is known to be a strong indicator of underlying human intentions and goals during manipulation tasks. This work studies gaze patterns of human teachers demonstrating tasks to robots and proposes ways in which such patterns can be used to enhance robot learning. Using both kinesthetic teaching and video demonstrations, we identify novel intention-revealing gaze behaviors during teaching. These prove to be informative in a variety of problems ranging from reference frame inference to segmentation of multi-step tasks. Based on our findings, we propose two proof-of-concept algorithms which show that gaze data can enhance subtask classification for a multi-step task up to 6% and reward inference and policy learning for a single-step task up to 67%. Our findings provide a foundation for a model of natural human gaze in robot learning from demonstration settings and present open problems for utilizing human gaze to enhance robot learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v100-saran20a, title = {Understanding Teacher Gaze Patterns for Robot Learning}, author = {Saran, Akanksha and Short, Elaine Schaertl and Thomaz, Andrea and Niekum, Scott}, booktitle = {Proceedings of the Conference on Robot Learning}, pages = {1247--1258}, year = {2020}, editor = {Kaelbling, Leslie Pack and Kragic, Danica and Sugiura, Komei}, volume = {100}, series = {Proceedings of Machine Learning Research}, month = {30 Oct--01 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v100/saran20a/saran20a.pdf}, url = {https://proceedings.mlr.press/v100/saran20a.html}, abstract = {Human gaze is known to be a strong indicator of underlying human intentions and goals during manipulation tasks. This work studies gaze patterns of human teachers demonstrating tasks to robots and proposes ways in which such patterns can be used to enhance robot learning. Using both kinesthetic teaching and video demonstrations, we identify novel intention-revealing gaze behaviors during teaching. These prove to be informative in a variety of problems ranging from reference frame inference to segmentation of multi-step tasks. Based on our findings, we propose two proof-of-concept algorithms which show that gaze data can enhance subtask classification for a multi-step task up to 6% and reward inference and policy learning for a single-step task up to 67%. Our findings provide a foundation for a model of natural human gaze in robot learning from demonstration settings and present open problems for utilizing human gaze to enhance robot learning.} }
Endnote
%0 Conference Paper %T Understanding Teacher Gaze Patterns for Robot Learning %A Akanksha Saran %A Elaine Schaertl Short %A Andrea Thomaz %A Scott Niekum %B Proceedings of the Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2020 %E Leslie Pack Kaelbling %E Danica Kragic %E Komei Sugiura %F pmlr-v100-saran20a %I PMLR %P 1247--1258 %U https://proceedings.mlr.press/v100/saran20a.html %V 100 %X Human gaze is known to be a strong indicator of underlying human intentions and goals during manipulation tasks. This work studies gaze patterns of human teachers demonstrating tasks to robots and proposes ways in which such patterns can be used to enhance robot learning. Using both kinesthetic teaching and video demonstrations, we identify novel intention-revealing gaze behaviors during teaching. These prove to be informative in a variety of problems ranging from reference frame inference to segmentation of multi-step tasks. Based on our findings, we propose two proof-of-concept algorithms which show that gaze data can enhance subtask classification for a multi-step task up to 6% and reward inference and policy learning for a single-step task up to 67%. Our findings provide a foundation for a model of natural human gaze in robot learning from demonstration settings and present open problems for utilizing human gaze to enhance robot learning.
APA
Saran, A., Short, E.S., Thomaz, A. & Niekum, S.. (2020). Understanding Teacher Gaze Patterns for Robot Learning. Proceedings of the Conference on Robot Learning, in Proceedings of Machine Learning Research 100:1247-1258 Available from https://proceedings.mlr.press/v100/saran20a.html.

Related Material