Open-TeleVision: Teleoperation with Immersive Active Visual Feedback

Xuxin Cheng, Jialong Li, Shiqi Yang, Ge Yang, Xiaolong Wang
Proceedings of The 8th Conference on Robot Learning, PMLR 270:2729-2749, 2025.

Abstract

Teleoperation serves as a powerful method for collecting on-robot data essential for robot learning from demonstrations. The intuitiveness and ease of use of the teleoperation system are crucial for ensuring high-quality, diverse, and scalable data. To achieve this, we propose an immersive teleoperation system Open-TeleVision that allows operators to actively perceive the robot’s surroundings in a stereoscopic manner. Additionally, the system mirrors the operator’s arm and hand movements on the robot, creating an immersive experience as if the operator’s mind is transmitted to a robot embodiment. We validate the effectiveness of our system by collecting data and training imitation learning policies on four long-horizon, precise tasks (can sorting, can insertion, folding, and unloading) for 2 different humanoid robots and deploy them in the real world. The entire system will be open-sourced.

Cite this Paper

BibTeX
@InProceedings{pmlr-v270-cheng25b, title = {Open-TeleVision: Teleoperation with Immersive Active Visual Feedback}, author = {Cheng, Xuxin and Li, Jialong and Yang, Shiqi and Yang, Ge and Wang, Xiaolong}, booktitle = {Proceedings of The 8th Conference on Robot Learning}, pages = {2729--2749}, year = {2025}, editor = {Agrawal, Pulkit and Kroemer, Oliver and Burgard, Wolfram}, volume = {270}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v270/main/assets/cheng25b/cheng25b.pdf}, url = {https://proceedings.mlr.press/v270/cheng25b.html}, abstract = {Teleoperation serves as a powerful method for collecting on-robot data essential for robot learning from demonstrations. The intuitiveness and ease of use of the teleoperation system are crucial for ensuring high-quality, diverse, and scalable data. To achieve this, we propose an immersive teleoperation system $\textbf{Open-TeleVision}$ that allows operators to actively perceive the robot’s surroundings in a stereoscopic manner. Additionally, the system mirrors the operator’s arm and hand movements on the robot, creating an immersive experience as if the operator’s mind is transmitted to a robot embodiment. We validate the effectiveness of our system by collecting data and training imitation learning policies on four long-horizon, precise tasks (can sorting, can insertion, folding, and unloading) for 2 different humanoid robots and deploy them in the real world. The entire system will be open-sourced.} }
Endnote
%0 Conference Paper %T Open-TeleVision: Teleoperation with Immersive Active Visual Feedback %A Xuxin Cheng %A Jialong Li %A Shiqi Yang %A Ge Yang %A Xiaolong Wang %B Proceedings of The 8th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Pulkit Agrawal %E Oliver Kroemer %E Wolfram Burgard %F pmlr-v270-cheng25b %I PMLR %P 2729--2749 %U https://proceedings.mlr.press/v270/cheng25b.html %V 270 %X Teleoperation serves as a powerful method for collecting on-robot data essential for robot learning from demonstrations. The intuitiveness and ease of use of the teleoperation system are crucial for ensuring high-quality, diverse, and scalable data. To achieve this, we propose an immersive teleoperation system $\textbf{Open-TeleVision}$ that allows operators to actively perceive the robot’s surroundings in a stereoscopic manner. Additionally, the system mirrors the operator’s arm and hand movements on the robot, creating an immersive experience as if the operator’s mind is transmitted to a robot embodiment. We validate the effectiveness of our system by collecting data and training imitation learning policies on four long-horizon, precise tasks (can sorting, can insertion, folding, and unloading) for 2 different humanoid robots and deploy them in the real world. The entire system will be open-sourced.
APA
Cheng, X., Li, J., Yang, S., Yang, G. & Wang, X.. (2025). Open-TeleVision: Teleoperation with Immersive Active Visual Feedback. Proceedings of The 8th Conference on Robot Learning, in Proceedings of Machine Learning Research 270:2729-2749 Available from https://proceedings.mlr.press/v270/cheng25b.html.

Related Material