Learning rich touch representations through cross-modal self-supervision

Martina Zambelli, Yusuf Aytar, Francesco Visin, Yuxiang Zhou, Raia Hadsell
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:1415-1425, 2021.

Abstract

The sense of touch is fundamental in several manipulation tasks, but rarely used in robot manipulation. In this work we tackle the problem of learning rich touch features from cross-modal self-supervision. We evaluate them identifying objects and their properties in a few-shot classification setting. Two new datasets are introduced using a simulated anthropomorphic robotic hand equipped with tactile sensors on both synthetic and daily life objects. Several self-supervised learning methods are benchmarked on these datasets, by evaluating few-shot classification on unseen objects and poses. Our experiments indicate that cross-modal self-supervision effectively improves touch representation, and in turn has great potential to enhance robot manipulation skills.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-zambelli21a, title = {Learning rich touch representations through cross-modal self-supervision}, author = {Zambelli, Martina and Aytar, Yusuf and Visin, Francesco and Zhou, Yuxiang and Hadsell, Raia}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {1415--1425}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/zambelli21a/zambelli21a.pdf}, url = {https://proceedings.mlr.press/v155/zambelli21a.html}, abstract = {The sense of touch is fundamental in several manipulation tasks, but rarely used in robot manipulation. In this work we tackle the problem of learning rich touch features from cross-modal self-supervision. We evaluate them identifying objects and their properties in a few-shot classification setting. Two new datasets are introduced using a simulated anthropomorphic robotic hand equipped with tactile sensors on both synthetic and daily life objects. Several self-supervised learning methods are benchmarked on these datasets, by evaluating few-shot classification on unseen objects and poses. Our experiments indicate that cross-modal self-supervision effectively improves touch representation, and in turn has great potential to enhance robot manipulation skills.} }
Endnote
%0 Conference Paper %T Learning rich touch representations through cross-modal self-supervision %A Martina Zambelli %A Yusuf Aytar %A Francesco Visin %A Yuxiang Zhou %A Raia Hadsell %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-zambelli21a %I PMLR %P 1415--1425 %U https://proceedings.mlr.press/v155/zambelli21a.html %V 155 %X The sense of touch is fundamental in several manipulation tasks, but rarely used in robot manipulation. In this work we tackle the problem of learning rich touch features from cross-modal self-supervision. We evaluate them identifying objects and their properties in a few-shot classification setting. Two new datasets are introduced using a simulated anthropomorphic robotic hand equipped with tactile sensors on both synthetic and daily life objects. Several self-supervised learning methods are benchmarked on these datasets, by evaluating few-shot classification on unseen objects and poses. Our experiments indicate that cross-modal self-supervision effectively improves touch representation, and in turn has great potential to enhance robot manipulation skills.
APA
Zambelli, M., Aytar, Y., Visin, F., Zhou, Y. & Hadsell, R.. (2021). Learning rich touch representations through cross-modal self-supervision. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:1415-1425 Available from https://proceedings.mlr.press/v155/zambelli21a.html.

Related Material