Robust Imitation Learning against Variations in Environment Dynamics

Jongseong Chae, Seungyul Han, Whiyoung Jung, Myungsik Cho, Sungho Choi, Youngchul Sung
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:2828-2852, 2022.

Abstract

In this paper, we propose a robust imitation learning (IL) framework that improves the robustness of IL when environment dynamics are perturbed. The existing IL framework trained in a single environment can catastrophically fail with perturbations in environment dynamics because it does not capture the situation that underlying environment dynamics can be changed. Our framework effectively deals with environments with varying dynamics by imitating multiple experts in sampled environment dynamics to enhance the robustness in general variations in environment dynamics. In order to robustly imitate the multiple sample experts, we minimize the risk with respect to the Jensen-Shannon divergence between the agent’s policy and each of the sample experts. Numerical results show that our algorithm significantly improves robustness against dynamics perturbations compared to conventional IL baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-chae22a, title = {Robust Imitation Learning against Variations in Environment Dynamics}, author = {Chae, Jongseong and Han, Seungyul and Jung, Whiyoung and Cho, Myungsik and Choi, Sungho and Sung, Youngchul}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {2828--2852}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/chae22a/chae22a.pdf}, url = {https://proceedings.mlr.press/v162/chae22a.html}, abstract = {In this paper, we propose a robust imitation learning (IL) framework that improves the robustness of IL when environment dynamics are perturbed. The existing IL framework trained in a single environment can catastrophically fail with perturbations in environment dynamics because it does not capture the situation that underlying environment dynamics can be changed. Our framework effectively deals with environments with varying dynamics by imitating multiple experts in sampled environment dynamics to enhance the robustness in general variations in environment dynamics. In order to robustly imitate the multiple sample experts, we minimize the risk with respect to the Jensen-Shannon divergence between the agent’s policy and each of the sample experts. Numerical results show that our algorithm significantly improves robustness against dynamics perturbations compared to conventional IL baselines.} }
Endnote
%0 Conference Paper %T Robust Imitation Learning against Variations in Environment Dynamics %A Jongseong Chae %A Seungyul Han %A Whiyoung Jung %A Myungsik Cho %A Sungho Choi %A Youngchul Sung %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-chae22a %I PMLR %P 2828--2852 %U https://proceedings.mlr.press/v162/chae22a.html %V 162 %X In this paper, we propose a robust imitation learning (IL) framework that improves the robustness of IL when environment dynamics are perturbed. The existing IL framework trained in a single environment can catastrophically fail with perturbations in environment dynamics because it does not capture the situation that underlying environment dynamics can be changed. Our framework effectively deals with environments with varying dynamics by imitating multiple experts in sampled environment dynamics to enhance the robustness in general variations in environment dynamics. In order to robustly imitate the multiple sample experts, we minimize the risk with respect to the Jensen-Shannon divergence between the agent’s policy and each of the sample experts. Numerical results show that our algorithm significantly improves robustness against dynamics perturbations compared to conventional IL baselines.
APA
Chae, J., Han, S., Jung, W., Cho, M., Choi, S. & Sung, Y.. (2022). Robust Imitation Learning against Variations in Environment Dynamics. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:2828-2852 Available from https://proceedings.mlr.press/v162/chae22a.html.

Related Material