HOI4ABOT: Human-Object Interaction Anticipation for Human Intention Reading Collaborative roBOTs

Esteve Valls Mascaro, Daniel Sliwowski, Dongheui Lee
Proceedings of The 7th Conference on Robot Learning, PMLR 229:1111-1130, 2023.

Abstract

Robots are becoming increasingly integrated into our lives, assisting us in various tasks. To ensure effective collaboration between humans and robots, it is essential that they understand our intentions and anticipate our actions. In this paper, we propose a Human-Object Interaction (HOI) anticipation framework for collaborative robots. We propose an efficient and robust transformer-based model to detect and anticipate HOIs from videos. This enhanced anticipation empowers robots to proactively assist humans, resulting in more efficient and intuitive collaborations. Our model outperforms state-of-the-art results in HOI detection and anticipation in VidHOI dataset with an increase of $1.76%$ and $1.04%$ in mAP respectively while being 15.4 times faster. We showcase the effectiveness of our approach through experimental results in a real robot, demonstrating that the robot’s ability to anticipate HOIs is key for better Human-Robot Interaction.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-mascaro23a, title = {HOI4ABOT: Human-Object Interaction Anticipation for Human Intention Reading Collaborative roBOTs}, author = {Mascaro, Esteve Valls and Sliwowski, Daniel and Lee, Dongheui}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {1111--1130}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/mascaro23a/mascaro23a.pdf}, url = {https://proceedings.mlr.press/v229/mascaro23a.html}, abstract = {Robots are becoming increasingly integrated into our lives, assisting us in various tasks. To ensure effective collaboration between humans and robots, it is essential that they understand our intentions and anticipate our actions. In this paper, we propose a Human-Object Interaction (HOI) anticipation framework for collaborative robots. We propose an efficient and robust transformer-based model to detect and anticipate HOIs from videos. This enhanced anticipation empowers robots to proactively assist humans, resulting in more efficient and intuitive collaborations. Our model outperforms state-of-the-art results in HOI detection and anticipation in VidHOI dataset with an increase of $1.76%$ and $1.04%$ in mAP respectively while being 15.4 times faster. We showcase the effectiveness of our approach through experimental results in a real robot, demonstrating that the robot’s ability to anticipate HOIs is key for better Human-Robot Interaction.} }
Endnote
%0 Conference Paper %T HOI4ABOT: Human-Object Interaction Anticipation for Human Intention Reading Collaborative roBOTs %A Esteve Valls Mascaro %A Daniel Sliwowski %A Dongheui Lee %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-mascaro23a %I PMLR %P 1111--1130 %U https://proceedings.mlr.press/v229/mascaro23a.html %V 229 %X Robots are becoming increasingly integrated into our lives, assisting us in various tasks. To ensure effective collaboration between humans and robots, it is essential that they understand our intentions and anticipate our actions. In this paper, we propose a Human-Object Interaction (HOI) anticipation framework for collaborative robots. We propose an efficient and robust transformer-based model to detect and anticipate HOIs from videos. This enhanced anticipation empowers robots to proactively assist humans, resulting in more efficient and intuitive collaborations. Our model outperforms state-of-the-art results in HOI detection and anticipation in VidHOI dataset with an increase of $1.76%$ and $1.04%$ in mAP respectively while being 15.4 times faster. We showcase the effectiveness of our approach through experimental results in a real robot, demonstrating that the robot’s ability to anticipate HOIs is key for better Human-Robot Interaction.
APA
Mascaro, E.V., Sliwowski, D. & Lee, D.. (2023). HOI4ABOT: Human-Object Interaction Anticipation for Human Intention Reading Collaborative roBOTs. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:1111-1130 Available from https://proceedings.mlr.press/v229/mascaro23a.html.

Related Material