DEFT: Dexterous Fine-Tuning for Hand Policies

Aditya Kannan, Kenneth Shaw, Shikhar Bahl, Pragna Mannam, Deepak Pathak
Proceedings of The 7th Conference on Robot Learning, PMLR 229:928-942, 2023.

Abstract

Dexterity is often seen as a cornerstone of complex manipulation. Humans are able to perform a host of skills with their hands, from making food to operating tools. In this paper, we investigate these challenges, especially in the case of soft, deformable objects as well as complex, relatively long-horizon tasks. Although, learning such behaviors from scratch can be data inefficient. To circumvent this, we propose a novel approach, DEFT (DExterous Fine-Tuning for Hand Policies), that leverages human-driven priors, which are executed directly in the real world. In order to improve upon these priors, DEFT involves an efficient online optimization procedure. With the integration of human-based learning and online fine-tuning, coupled with a soft robotic hand, DEFT demonstrates success across various tasks, establishing a robust, data-efficient pathway toward general dexterous manipulation. Please see our website at https://dexterousfinetuning.github.io for video results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-kannan23a, title = {DEFT: Dexterous Fine-Tuning for Hand Policies}, author = {Kannan, Aditya and Shaw, Kenneth and Bahl, Shikhar and Mannam, Pragna and Pathak, Deepak}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {928--942}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/kannan23a/kannan23a.pdf}, url = {https://proceedings.mlr.press/v229/kannan23a.html}, abstract = {Dexterity is often seen as a cornerstone of complex manipulation. Humans are able to perform a host of skills with their hands, from making food to operating tools. In this paper, we investigate these challenges, especially in the case of soft, deformable objects as well as complex, relatively long-horizon tasks. Although, learning such behaviors from scratch can be data inefficient. To circumvent this, we propose a novel approach, DEFT (DExterous Fine-Tuning for Hand Policies), that leverages human-driven priors, which are executed directly in the real world. In order to improve upon these priors, DEFT involves an efficient online optimization procedure. With the integration of human-based learning and online fine-tuning, coupled with a soft robotic hand, DEFT demonstrates success across various tasks, establishing a robust, data-efficient pathway toward general dexterous manipulation. Please see our website at https://dexterousfinetuning.github.io for video results.} }
Endnote
%0 Conference Paper %T DEFT: Dexterous Fine-Tuning for Hand Policies %A Aditya Kannan %A Kenneth Shaw %A Shikhar Bahl %A Pragna Mannam %A Deepak Pathak %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-kannan23a %I PMLR %P 928--942 %U https://proceedings.mlr.press/v229/kannan23a.html %V 229 %X Dexterity is often seen as a cornerstone of complex manipulation. Humans are able to perform a host of skills with their hands, from making food to operating tools. In this paper, we investigate these challenges, especially in the case of soft, deformable objects as well as complex, relatively long-horizon tasks. Although, learning such behaviors from scratch can be data inefficient. To circumvent this, we propose a novel approach, DEFT (DExterous Fine-Tuning for Hand Policies), that leverages human-driven priors, which are executed directly in the real world. In order to improve upon these priors, DEFT involves an efficient online optimization procedure. With the integration of human-based learning and online fine-tuning, coupled with a soft robotic hand, DEFT demonstrates success across various tasks, establishing a robust, data-efficient pathway toward general dexterous manipulation. Please see our website at https://dexterousfinetuning.github.io for video results.
APA
Kannan, A., Shaw, K., Bahl, S., Mannam, P. & Pathak, D.. (2023). DEFT: Dexterous Fine-Tuning for Hand Policies. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:928-942 Available from https://proceedings.mlr.press/v229/kannan23a.html.

Related Material