Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks

Ji Woong Kim, Tony Z. Zhao, Samuel Schmidgall, Anton Deguet, Marin Kobilarov, Chelsea Finn, Axel Krieger
Proceedings of The 8th Conference on Robot Learning, PMLR 270:130-144, 2025.

Abstract

We explore whether surgical manipulation tasks can be learned on the da Vinci robot via imitation learning. However, the da Vinci system presents unique challenges which hinder straight-forward implementation of imitation learning. Notably, its forward kinematics is inconsistent due to imprecise joint measurements, and naively training a policy using such approximate kinematics data often leads to task failure. To overcome this limitation, we introduce a relative action formulation which enables successful policy training and deployment using its approximate kinematics data. A promising outcome of this approach is that the large repository of clinical data, which contains approximate kinematics, may be directly utilized for robot learning without further corrections. We demonstrate our findings through successful execution of three fundamental surgical tasks, including tissue manipulation, needle handling, and knot-tying.

Cite this Paper

BibTeX
@InProceedings{pmlr-v270-kim25a, title = {Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks}, author = {Kim, Ji Woong and Zhao, Tony Z. and Schmidgall, Samuel and Deguet, Anton and Kobilarov, Marin and Finn, Chelsea and Krieger, Axel}, booktitle = {Proceedings of The 8th Conference on Robot Learning}, pages = {130--144}, year = {2025}, editor = {Agrawal, Pulkit and Kroemer, Oliver and Burgard, Wolfram}, volume = {270}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v270/main/assets/kim25a/kim25a.pdf}, url = {https://proceedings.mlr.press/v270/kim25a.html}, abstract = {We explore whether surgical manipulation tasks can be learned on the da Vinci robot via imitation learning. However, the da Vinci system presents unique challenges which hinder straight-forward implementation of imitation learning. Notably, its forward kinematics is inconsistent due to imprecise joint measurements, and naively training a policy using such approximate kinematics data often leads to task failure. To overcome this limitation, we introduce a relative action formulation which enables successful policy training and deployment using its approximate kinematics data. A promising outcome of this approach is that the large repository of clinical data, which contains approximate kinematics, may be directly utilized for robot learning without further corrections. We demonstrate our findings through successful execution of three fundamental surgical tasks, including tissue manipulation, needle handling, and knot-tying.} }
Endnote
%0 Conference Paper %T Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks %A Ji Woong Kim %A Tony Z. Zhao %A Samuel Schmidgall %A Anton Deguet %A Marin Kobilarov %A Chelsea Finn %A Axel Krieger %B Proceedings of The 8th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Pulkit Agrawal %E Oliver Kroemer %E Wolfram Burgard %F pmlr-v270-kim25a %I PMLR %P 130--144 %U https://proceedings.mlr.press/v270/kim25a.html %V 270 %X We explore whether surgical manipulation tasks can be learned on the da Vinci robot via imitation learning. However, the da Vinci system presents unique challenges which hinder straight-forward implementation of imitation learning. Notably, its forward kinematics is inconsistent due to imprecise joint measurements, and naively training a policy using such approximate kinematics data often leads to task failure. To overcome this limitation, we introduce a relative action formulation which enables successful policy training and deployment using its approximate kinematics data. A promising outcome of this approach is that the large repository of clinical data, which contains approximate kinematics, may be directly utilized for robot learning without further corrections. We demonstrate our findings through successful execution of three fundamental surgical tasks, including tissue manipulation, needle handling, and knot-tying.
APA
Kim, J.W., Zhao, T.Z., Schmidgall, S., Deguet, A., Kobilarov, M., Finn, C. & Krieger, A.. (2025). Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks. Proceedings of The 8th Conference on Robot Learning, in Proceedings of Machine Learning Research 270:130-144 Available from https://proceedings.mlr.press/v270/kim25a.html.

Related Material