SeMAIL: Eliminating Distractors in Visual Imitation via Separated Models

Shenghua Wan, Yucen Wang, Minghao Shao, Ruying Chen, De-Chuan Zhan
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:35426-35443, 2023.

Abstract

Model-based imitation learning (MBIL) is a popular reinforcement learning method that improves sample efficiency on high-dimension input sources, such as images and videos. Following the convention of MBIL research, existing algorithms are highly deceptive by task-irrelevant information, especially moving distractors in videos. To tackle this problem, we propose a new algorithm - named Separated Model-based Adversarial Imitation Learning (SeMAIL) - decoupling the environment dynamics into two parts by task-relevant dependency, which is determined by agent actions, and training separately. In this way, the agent can imagine its trajectories and imitate the expert behavior efficiently in task-relevant state space. Our method achieves near-expert performance on various visual control tasks with complex observations and the more challenging tasks with different backgrounds from expert observations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-wan23c, title = {{S}e{MAIL}: Eliminating Distractors in Visual Imitation via Separated Models}, author = {Wan, Shenghua and Wang, Yucen and Shao, Minghao and Chen, Ruying and Zhan, De-Chuan}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {35426--35443}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/wan23c/wan23c.pdf}, url = {https://proceedings.mlr.press/v202/wan23c.html}, abstract = {Model-based imitation learning (MBIL) is a popular reinforcement learning method that improves sample efficiency on high-dimension input sources, such as images and videos. Following the convention of MBIL research, existing algorithms are highly deceptive by task-irrelevant information, especially moving distractors in videos. To tackle this problem, we propose a new algorithm - named Separated Model-based Adversarial Imitation Learning (SeMAIL) - decoupling the environment dynamics into two parts by task-relevant dependency, which is determined by agent actions, and training separately. In this way, the agent can imagine its trajectories and imitate the expert behavior efficiently in task-relevant state space. Our method achieves near-expert performance on various visual control tasks with complex observations and the more challenging tasks with different backgrounds from expert observations.} }
Endnote
%0 Conference Paper %T SeMAIL: Eliminating Distractors in Visual Imitation via Separated Models %A Shenghua Wan %A Yucen Wang %A Minghao Shao %A Ruying Chen %A De-Chuan Zhan %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-wan23c %I PMLR %P 35426--35443 %U https://proceedings.mlr.press/v202/wan23c.html %V 202 %X Model-based imitation learning (MBIL) is a popular reinforcement learning method that improves sample efficiency on high-dimension input sources, such as images and videos. Following the convention of MBIL research, existing algorithms are highly deceptive by task-irrelevant information, especially moving distractors in videos. To tackle this problem, we propose a new algorithm - named Separated Model-based Adversarial Imitation Learning (SeMAIL) - decoupling the environment dynamics into two parts by task-relevant dependency, which is determined by agent actions, and training separately. In this way, the agent can imagine its trajectories and imitate the expert behavior efficiently in task-relevant state space. Our method achieves near-expert performance on various visual control tasks with complex observations and the more challenging tasks with different backgrounds from expert observations.
APA
Wan, S., Wang, Y., Shao, M., Chen, R. & Zhan, D.. (2023). SeMAIL: Eliminating Distractors in Visual Imitation via Separated Models. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:35426-35443 Available from https://proceedings.mlr.press/v202/wan23c.html.

Related Material