Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations

Chenhao Li, Marin Vlastelica, Sebastian Blaes, Jonas Frey, Felix Grimminger, Georg Martius
Proceedings of The 6th Conference on Robot Learning, PMLR 205:342-352, 2023.

Abstract

Learning agile skills is one of the main challenges in robotics. To this end, reinforcement learning approaches have achieved impressive results. These methods require explicit task information in terms of a reward function or an expert that can be queried in simulation to provide a target control output, which limits their applicability. In this work, we propose a generative adversarial method for inferring reward functions from partial and potentially physically incompatible demonstrations for successful skill acquirement where reference or expert demonstrations are not easily accessible. Moreover, we show that by using a Wasserstein GAN formulation and transitions from demonstrations with rough and partial information as input, we are able to extract policies that are robust and capable of imitating demonstrated behaviors. Finally, the obtained skills such as a backflip are tested on an agile quadruped robot called Solo 8 and present faithful replication of hand-held human demonstrations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-li23b, title = {Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations}, author = {Li, Chenhao and Vlastelica, Marin and Blaes, Sebastian and Frey, Jonas and Grimminger, Felix and Martius, Georg}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {342--352}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/li23b/li23b.pdf}, url = {https://proceedings.mlr.press/v205/li23b.html}, abstract = {Learning agile skills is one of the main challenges in robotics. To this end, reinforcement learning approaches have achieved impressive results. These methods require explicit task information in terms of a reward function or an expert that can be queried in simulation to provide a target control output, which limits their applicability. In this work, we propose a generative adversarial method for inferring reward functions from partial and potentially physically incompatible demonstrations for successful skill acquirement where reference or expert demonstrations are not easily accessible. Moreover, we show that by using a Wasserstein GAN formulation and transitions from demonstrations with rough and partial information as input, we are able to extract policies that are robust and capable of imitating demonstrated behaviors. Finally, the obtained skills such as a backflip are tested on an agile quadruped robot called Solo 8 and present faithful replication of hand-held human demonstrations.} }
Endnote
%0 Conference Paper %T Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations %A Chenhao Li %A Marin Vlastelica %A Sebastian Blaes %A Jonas Frey %A Felix Grimminger %A Georg Martius %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-li23b %I PMLR %P 342--352 %U https://proceedings.mlr.press/v205/li23b.html %V 205 %X Learning agile skills is one of the main challenges in robotics. To this end, reinforcement learning approaches have achieved impressive results. These methods require explicit task information in terms of a reward function or an expert that can be queried in simulation to provide a target control output, which limits their applicability. In this work, we propose a generative adversarial method for inferring reward functions from partial and potentially physically incompatible demonstrations for successful skill acquirement where reference or expert demonstrations are not easily accessible. Moreover, we show that by using a Wasserstein GAN formulation and transitions from demonstrations with rough and partial information as input, we are able to extract policies that are robust and capable of imitating demonstrated behaviors. Finally, the obtained skills such as a backflip are tested on an agile quadruped robot called Solo 8 and present faithful replication of hand-held human demonstrations.
APA
Li, C., Vlastelica, M., Blaes, S., Frey, J., Grimminger, F. & Martius, G.. (2023). Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:342-352 Available from https://proceedings.mlr.press/v205/li23b.html.

Related Material