Learning Eye-in-Hand Camera Calibration from a Single Image

Eugene Valassakis, Kamil Dreczkowski, Edward Johns
Proceedings of the 5th Conference on Robot Learning, PMLR 164:1336-1346, 2022.

Abstract

Eye-in-hand camera calibration is a fundamental and long-studied problem in robotics. We present a study on using learning-based methods for solving this problem online from a single RGB image, whilst training our models with entirely synthetic data. We study three main approaches: one direct regression model that directly predicts the extrinsic matrix from an image, one sparse correspondence model that regresses 2D keypoints and then uses PnP, and one dense correspondence model that uses regressed depth and segmentation maps to enable ICP pose estimation. In our experiments, we benchmark these methods against each other and against well-established classical methods, to find the surprising result that direct regression outperforms other approaches, and we perform noise-sensitivity analysis to gain further insights into these results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-valassakis22a, title = {Learning Eye-in-Hand Camera Calibration from a Single Image}, author = {Valassakis, Eugene and Dreczkowski, Kamil and Johns, Edward}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {1336--1346}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/valassakis22a/valassakis22a.pdf}, url = {https://proceedings.mlr.press/v164/valassakis22a.html}, abstract = {Eye-in-hand camera calibration is a fundamental and long-studied problem in robotics. We present a study on using learning-based methods for solving this problem online from a single RGB image, whilst training our models with entirely synthetic data. We study three main approaches: one direct regression model that directly predicts the extrinsic matrix from an image, one sparse correspondence model that regresses 2D keypoints and then uses PnP, and one dense correspondence model that uses regressed depth and segmentation maps to enable ICP pose estimation. In our experiments, we benchmark these methods against each other and against well-established classical methods, to find the surprising result that direct regression outperforms other approaches, and we perform noise-sensitivity analysis to gain further insights into these results.} }
Endnote
%0 Conference Paper %T Learning Eye-in-Hand Camera Calibration from a Single Image %A Eugene Valassakis %A Kamil Dreczkowski %A Edward Johns %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-valassakis22a %I PMLR %P 1336--1346 %U https://proceedings.mlr.press/v164/valassakis22a.html %V 164 %X Eye-in-hand camera calibration is a fundamental and long-studied problem in robotics. We present a study on using learning-based methods for solving this problem online from a single RGB image, whilst training our models with entirely synthetic data. We study three main approaches: one direct regression model that directly predicts the extrinsic matrix from an image, one sparse correspondence model that regresses 2D keypoints and then uses PnP, and one dense correspondence model that uses regressed depth and segmentation maps to enable ICP pose estimation. In our experiments, we benchmark these methods against each other and against well-established classical methods, to find the surprising result that direct regression outperforms other approaches, and we perform noise-sensitivity analysis to gain further insights into these results.
APA
Valassakis, E., Dreczkowski, K. & Johns, E.. (2022). Learning Eye-in-Hand Camera Calibration from a Single Image. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:1336-1346 Available from https://proceedings.mlr.press/v164/valassakis22a.html.

Related Material