exUMI: Extensible Robot Teaching System with Action-aware Task-agnostic Tactile Representation

Yue Xu, Litao Wei, Pengyu An, Qingyu Zhang, Yong-Lu Li
Proceedings of The 9th Conference on Robot Learning, PMLR 305:2536-2554, 2025.

Abstract

Tactile-aware robot learning faces critical challenges in data collection and representation due to data scarcity and sparsity, and the absence of force feedback in existing systems. To address these limitations, we introduce a tactile robot learning system with both hardware and algorithm innovations. We present exUMI, an extensible data collection device that enhances the vanilla UMI with robust proprioception (via AR MoCap and rotary encoder), modular visuo-tactile sensing, and automated calibration, achieving 100% data usability. Building on an efficient collection of over 1 M tactile frames, we propose Tactile Prediction Pretraining (TPP), a representation learning framework through action-aware temporal tactile prediction, capturing contact dynamics and mitigates tactile sparsity. Real-world experiments show that TPP outperforms traditional tactile imitation learning. Our work bridges the gap between human tactile intuition and robot learning through co-designed hardware and algorithms, offering open-source resources to advance contact-rich manipulation research.

Cite this Paper

BibTeX
@InProceedings{pmlr-v305-xu25e, title = {exUMI: Extensible Robot Teaching System with Action-aware Task-agnostic Tactile Representation}, author = {Xu, Yue and Wei, Litao and An, Pengyu and Zhang, Qingyu and Li, Yong-Lu}, booktitle = {Proceedings of The 9th Conference on Robot Learning}, pages = {2536--2554}, year = {2025}, editor = {Lim, Joseph and Song, Shuran and Park, Hae-Won}, volume = {305}, series = {Proceedings of Machine Learning Research}, month = {27--30 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v305/main/assets/xu25e/xu25e.pdf}, url = {https://proceedings.mlr.press/v305/xu25e.html}, abstract = {Tactile-aware robot learning faces critical challenges in data collection and representation due to data scarcity and sparsity, and the absence of force feedback in existing systems. To address these limitations, we introduce a tactile robot learning system with both hardware and algorithm innovations. We present exUMI, an extensible data collection device that enhances the vanilla UMI with robust proprioception (via AR MoCap and rotary encoder), modular visuo-tactile sensing, and automated calibration, achieving 100% data usability. Building on an efficient collection of over 1 M tactile frames, we propose Tactile Prediction Pretraining (TPP), a representation learning framework through action-aware temporal tactile prediction, capturing contact dynamics and mitigates tactile sparsity. Real-world experiments show that TPP outperforms traditional tactile imitation learning. Our work bridges the gap between human tactile intuition and robot learning through co-designed hardware and algorithms, offering open-source resources to advance contact-rich manipulation research.} }
Endnote
%0 Conference Paper %T exUMI: Extensible Robot Teaching System with Action-aware Task-agnostic Tactile Representation %A Yue Xu %A Litao Wei %A Pengyu An %A Qingyu Zhang %A Yong-Lu Li %B Proceedings of The 9th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Joseph Lim %E Shuran Song %E Hae-Won Park %F pmlr-v305-xu25e %I PMLR %P 2536--2554 %U https://proceedings.mlr.press/v305/xu25e.html %V 305 %X Tactile-aware robot learning faces critical challenges in data collection and representation due to data scarcity and sparsity, and the absence of force feedback in existing systems. To address these limitations, we introduce a tactile robot learning system with both hardware and algorithm innovations. We present exUMI, an extensible data collection device that enhances the vanilla UMI with robust proprioception (via AR MoCap and rotary encoder), modular visuo-tactile sensing, and automated calibration, achieving 100% data usability. Building on an efficient collection of over 1 M tactile frames, we propose Tactile Prediction Pretraining (TPP), a representation learning framework through action-aware temporal tactile prediction, capturing contact dynamics and mitigates tactile sparsity. Real-world experiments show that TPP outperforms traditional tactile imitation learning. Our work bridges the gap between human tactile intuition and robot learning through co-designed hardware and algorithms, offering open-source resources to advance contact-rich manipulation research.
APA
Xu, Y., Wei, L., An, P., Zhang, Q. & Li, Y.. (2025). exUMI: Extensible Robot Teaching System with Action-aware Task-agnostic Tactile Representation. Proceedings of The 9th Conference on Robot Learning, in Proceedings of Machine Learning Research 305:2536-2554 Available from https://proceedings.mlr.press/v305/xu25e.html.

Related Material