Learning Markerless Robot-Depth Camera Calibration and End-Effector Pose Estimation

Bugra Can Sefercik, Baris Akgun
Proceedings of The 6th Conference on Robot Learning, PMLR 205:1586-1595, 2023.

Abstract

Traditional approaches to extrinsic calibration use fiducial markers and learning-based approaches rely heavily on simulation data. In this work, we present a learning-based markerless extrinsic calibration system that uses a depth camera and does not rely on simulation data. We learn models for end-effector (EE) segmentation, single-frame rotation prediction and keypoint detection, from automatically generated real-world data. We use a transformation trick to get EE pose estimates from rotation predictions and a matching algorithm to get EE pose estimates from keypoint predictions. We further utilize the iterative closest point algorithm, multiple-frames, filtering and outlier detection to increase calibration robustness. Our evaluations with training data from multiple camera poses and test data from previously unseen poses give sub-centimeter and sub-deciradian average calibration and pose estimation errors. We also show that a carefully selected single training pose gives comparable results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-sefercik23a, title = {Learning Markerless Robot-Depth Camera Calibration and End-Effector Pose Estimation}, author = {Sefercik, Bugra Can and Akgun, Baris}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {1586--1595}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/sefercik23a/sefercik23a.pdf}, url = {https://proceedings.mlr.press/v205/sefercik23a.html}, abstract = {Traditional approaches to extrinsic calibration use fiducial markers and learning-based approaches rely heavily on simulation data. In this work, we present a learning-based markerless extrinsic calibration system that uses a depth camera and does not rely on simulation data. We learn models for end-effector (EE) segmentation, single-frame rotation prediction and keypoint detection, from automatically generated real-world data. We use a transformation trick to get EE pose estimates from rotation predictions and a matching algorithm to get EE pose estimates from keypoint predictions. We further utilize the iterative closest point algorithm, multiple-frames, filtering and outlier detection to increase calibration robustness. Our evaluations with training data from multiple camera poses and test data from previously unseen poses give sub-centimeter and sub-deciradian average calibration and pose estimation errors. We also show that a carefully selected single training pose gives comparable results.} }
Endnote
%0 Conference Paper %T Learning Markerless Robot-Depth Camera Calibration and End-Effector Pose Estimation %A Bugra Can Sefercik %A Baris Akgun %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-sefercik23a %I PMLR %P 1586--1595 %U https://proceedings.mlr.press/v205/sefercik23a.html %V 205 %X Traditional approaches to extrinsic calibration use fiducial markers and learning-based approaches rely heavily on simulation data. In this work, we present a learning-based markerless extrinsic calibration system that uses a depth camera and does not rely on simulation data. We learn models for end-effector (EE) segmentation, single-frame rotation prediction and keypoint detection, from automatically generated real-world data. We use a transformation trick to get EE pose estimates from rotation predictions and a matching algorithm to get EE pose estimates from keypoint predictions. We further utilize the iterative closest point algorithm, multiple-frames, filtering and outlier detection to increase calibration robustness. Our evaluations with training data from multiple camera poses and test data from previously unseen poses give sub-centimeter and sub-deciradian average calibration and pose estimation errors. We also show that a carefully selected single training pose gives comparable results.
APA
Sefercik, B.C. & Akgun, B.. (2023). Learning Markerless Robot-Depth Camera Calibration and End-Effector Pose Estimation. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:1586-1595 Available from https://proceedings.mlr.press/v205/sefercik23a.html.

Related Material