Semi-Supervised Haptic Material Recognition for Robots using Generative Adversarial Networks

Zackory Erickson, Sonia Chernova, Charles C. Kemp
Proceedings of the 1st Annual Conference on Robot Learning, PMLR 78:157-166, 2017.

Abstract

Material recognition enables robots to incorporate knowledge of material properties into their interactions with everyday objects. For example, material recognition opens up opportunities for clearer communication with a robot, such as "bring me the metal coffee mug", and recognizing plastic versus metal is crucial when using a microwave or oven. However, collecting labeled training data with a robot is often more difficult than unlabeled data. We present a semi-supervised learning approach for material recognition that uses generative adversarial networks (GANs) with haptic features such as force, temperature, and vibration. Our approach achieves state-of-the-art results and enables a robot to estimate the material class of household objects with  90% accuracy when 92% of the training data are unlabeled. We explore how well this approach can recognize the material of new objects and we discuss challenges facing generalization. To motivate learning from unlabeled training data, we also compare results against several common supervised learning classifiers. In addition, we have released the dataset used for this work which consists of time-series haptic measurements from a robot that conducted thousands of interactions with 72 household objects.

Cite this Paper

BibTeX
@InProceedings{pmlr-v78-erickson17a, title = {Semi-Supervised Haptic Material Recognition for Robots using Generative Adversarial Networks}, author = {Erickson, Zackory and Chernova, Sonia and Kemp, Charles C.}, booktitle = {Proceedings of the 1st Annual Conference on Robot Learning}, pages = {157--166}, year = {2017}, editor = {Levine, Sergey and Vanhoucke, Vincent and Goldberg, Ken}, volume = {78}, series = {Proceedings of Machine Learning Research}, month = {13--15 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v78/erickson17a/erickson17a.pdf}, url = {https://proceedings.mlr.press/v78/erickson17a.html}, abstract = {Material recognition enables robots to incorporate knowledge of material properties into their interactions with everyday objects. For example, material recognition opens up opportunities for clearer communication with a robot, such as "bring me the metal coffee mug", and recognizing plastic versus metal is crucial when using a microwave or oven. However, collecting labeled training data with a robot is often more difficult than unlabeled data. We present a semi-supervised learning approach for material recognition that uses generative adversarial networks (GANs) with haptic features such as force, temperature, and vibration. Our approach achieves state-of-the-art results and enables a robot to estimate the material class of household objects with  90% accuracy when 92% of the training data are unlabeled. We explore how well this approach can recognize the material of new objects and we discuss challenges facing generalization. To motivate learning from unlabeled training data, we also compare results against several common supervised learning classifiers. In addition, we have released the dataset used for this work which consists of time-series haptic measurements from a robot that conducted thousands of interactions with 72 household objects.} }
Endnote
%0 Conference Paper %T Semi-Supervised Haptic Material Recognition for Robots using Generative Adversarial Networks %A Zackory Erickson %A Sonia Chernova %A Charles C. Kemp %B Proceedings of the 1st Annual Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2017 %E Sergey Levine %E Vincent Vanhoucke %E Ken Goldberg %F pmlr-v78-erickson17a %I PMLR %P 157--166 %U https://proceedings.mlr.press/v78/erickson17a.html %V 78 %X Material recognition enables robots to incorporate knowledge of material properties into their interactions with everyday objects. For example, material recognition opens up opportunities for clearer communication with a robot, such as "bring me the metal coffee mug", and recognizing plastic versus metal is crucial when using a microwave or oven. However, collecting labeled training data with a robot is often more difficult than unlabeled data. We present a semi-supervised learning approach for material recognition that uses generative adversarial networks (GANs) with haptic features such as force, temperature, and vibration. Our approach achieves state-of-the-art results and enables a robot to estimate the material class of household objects with  90% accuracy when 92% of the training data are unlabeled. We explore how well this approach can recognize the material of new objects and we discuss challenges facing generalization. To motivate learning from unlabeled training data, we also compare results against several common supervised learning classifiers. In addition, we have released the dataset used for this work which consists of time-series haptic measurements from a robot that conducted thousands of interactions with 72 household objects.
APA
Erickson, Z., Chernova, S. & Kemp, C.C.. (2017). Semi-Supervised Haptic Material Recognition for Robots using Generative Adversarial Networks. Proceedings of the 1st Annual Conference on Robot Learning, in Proceedings of Machine Learning Research 78:157-166 Available from https://proceedings.mlr.press/v78/erickson17a.html.

Related Material