Interactive Imitation Learning in State-Space

Snehal Jauhri, Carlos Celemin, Jens Kober
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:682-692, 2021.

Abstract

Imitation Learning techniques enable programming the behaviour of agents through demonstrations rather than manual engineering. However, they are limited by the quality of available demonstration data. Interactive Imitation Learning techniques can improve the efficacy of learning since they involve teachers providing feedback while the agent executes its task. In this work, we propose a novel Interactive Learning technique that uses human feedback in state-space to train and improve agent behaviour (as opposed to alternative methods that use feedback in action-space). Our method titled Teaching Imitative Policies in State-space (TIPS) enables providing guidance to the agent in terms of ‘changing its state’ which is often more intuitive for a human demonstrator. Through continuous improvement via corrective feedback, agents trained by non-expert demonstrators using TIPS outperformed the demonstrator and conventional Imitation Learning agents.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-jauhri21a, title = {Interactive Imitation Learning in State-Space}, author = {Jauhri, Snehal and Celemin, Carlos and Kober, Jens}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {682--692}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/jauhri21a/jauhri21a.pdf}, url = {https://proceedings.mlr.press/v155/jauhri21a.html}, abstract = {Imitation Learning techniques enable programming the behaviour of agents through demonstrations rather than manual engineering. However, they are limited by the quality of available demonstration data. Interactive Imitation Learning techniques can improve the efficacy of learning since they involve teachers providing feedback while the agent executes its task. In this work, we propose a novel Interactive Learning technique that uses human feedback in state-space to train and improve agent behaviour (as opposed to alternative methods that use feedback in action-space). Our method titled Teaching Imitative Policies in State-space (TIPS) enables providing guidance to the agent in terms of ‘changing its state’ which is often more intuitive for a human demonstrator. Through continuous improvement via corrective feedback, agents trained by non-expert demonstrators using TIPS outperformed the demonstrator and conventional Imitation Learning agents.} }
Endnote
%0 Conference Paper %T Interactive Imitation Learning in State-Space %A Snehal Jauhri %A Carlos Celemin %A Jens Kober %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-jauhri21a %I PMLR %P 682--692 %U https://proceedings.mlr.press/v155/jauhri21a.html %V 155 %X Imitation Learning techniques enable programming the behaviour of agents through demonstrations rather than manual engineering. However, they are limited by the quality of available demonstration data. Interactive Imitation Learning techniques can improve the efficacy of learning since they involve teachers providing feedback while the agent executes its task. In this work, we propose a novel Interactive Learning technique that uses human feedback in state-space to train and improve agent behaviour (as opposed to alternative methods that use feedback in action-space). Our method titled Teaching Imitative Policies in State-space (TIPS) enables providing guidance to the agent in terms of ‘changing its state’ which is often more intuitive for a human demonstrator. Through continuous improvement via corrective feedback, agents trained by non-expert demonstrators using TIPS outperformed the demonstrator and conventional Imitation Learning agents.
APA
Jauhri, S., Celemin, C. & Kober, J.. (2021). Interactive Imitation Learning in State-Space. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:682-692 Available from https://proceedings.mlr.press/v155/jauhri21a.html.

Related Material