Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation

Pratyusha Sharma, Lekha Mohan, Lerrel Pinto, Abhinav Gupta
Proceedings of The 2nd Conference on Robot Learning, PMLR 87:906-915, 2018.

Abstract

In recent years, we have seen an emergence of data-driven approaches in robotics. However, most existing efforts and datasets are either in simulation or focus on a single task in isolation such as grasping, pushing or poking. In order to make progress and capture the space of manipulation, we would need to collect a large-scale dataset of diverse tasks such as pouring, opening bottles, stacking objects etc. But how does one collect such a dataset? In this paper, we present the largest available robotic-demonstration dataset (MIME) that contains 8260 human-robot demonstrations over 20 different robotic tasks2. These tasks range from the simple task of pushing objects to the difficult task of stacking household objects. Our dataset consists of videos of human demonstrations and kinesthetic trajectories of robot demonstrations. We also propose to use this dataset for the task of mapping 3rd person video features to robot trajectories. Furthermore, we present two different approaches using this dataset and evaluate the predicted robot trajectories against ground-truth trajectories. We hope our dataset inspires research in multiple areas including visual imitation, trajectory prediction and multi-task robotic learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v87-sharma18a, title = {Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation}, author = {Sharma, Pratyusha and Mohan, Lekha and Pinto, Lerrel and Gupta, Abhinav}, booktitle = {Proceedings of The 2nd Conference on Robot Learning}, pages = {906--915}, year = {2018}, editor = {Billard, Aude and Dragan, Anca and Peters, Jan and Morimoto, Jun}, volume = {87}, series = {Proceedings of Machine Learning Research}, month = {29--31 Oct}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v87/sharma18a/sharma18a.pdf}, url = {https://proceedings.mlr.press/v87/sharma18a.html}, abstract = {In recent years, we have seen an emergence of data-driven approaches in robotics. However, most existing efforts and datasets are either in simulation or focus on a single task in isolation such as grasping, pushing or poking. In order to make progress and capture the space of manipulation, we would need to collect a large-scale dataset of diverse tasks such as pouring, opening bottles, stacking objects etc. But how does one collect such a dataset? In this paper, we present the largest available robotic-demonstration dataset (MIME) that contains 8260 human-robot demonstrations over 20 different robotic tasks2. These tasks range from the simple task of pushing objects to the difficult task of stacking household objects. Our dataset consists of videos of human demonstrations and kinesthetic trajectories of robot demonstrations. We also propose to use this dataset for the task of mapping 3rd person video features to robot trajectories. Furthermore, we present two different approaches using this dataset and evaluate the predicted robot trajectories against ground-truth trajectories. We hope our dataset inspires research in multiple areas including visual imitation, trajectory prediction and multi-task robotic learning. } }
Endnote
%0 Conference Paper %T Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation %A Pratyusha Sharma %A Lekha Mohan %A Lerrel Pinto %A Abhinav Gupta %B Proceedings of The 2nd Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2018 %E Aude Billard %E Anca Dragan %E Jan Peters %E Jun Morimoto %F pmlr-v87-sharma18a %I PMLR %P 906--915 %U https://proceedings.mlr.press/v87/sharma18a.html %V 87 %X In recent years, we have seen an emergence of data-driven approaches in robotics. However, most existing efforts and datasets are either in simulation or focus on a single task in isolation such as grasping, pushing or poking. In order to make progress and capture the space of manipulation, we would need to collect a large-scale dataset of diverse tasks such as pouring, opening bottles, stacking objects etc. But how does one collect such a dataset? In this paper, we present the largest available robotic-demonstration dataset (MIME) that contains 8260 human-robot demonstrations over 20 different robotic tasks2. These tasks range from the simple task of pushing objects to the difficult task of stacking household objects. Our dataset consists of videos of human demonstrations and kinesthetic trajectories of robot demonstrations. We also propose to use this dataset for the task of mapping 3rd person video features to robot trajectories. Furthermore, we present two different approaches using this dataset and evaluate the predicted robot trajectories against ground-truth trajectories. We hope our dataset inspires research in multiple areas including visual imitation, trajectory prediction and multi-task robotic learning.
APA
Sharma, P., Mohan, L., Pinto, L. & Gupta, A.. (2018). Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation. Proceedings of The 2nd Conference on Robot Learning, in Proceedings of Machine Learning Research 87:906-915 Available from https://proceedings.mlr.press/v87/sharma18a.html.

Related Material