NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities

Ruohan Zhang, Sharon Lee, Minjune Hwang, Ayano Hiranaka, Chen Wang, Wensi Ai, Jin Jie Ryan Tan, Shreya Gupta, Yilun Hao, Gabrael Levine, Ruohan Gao, Anthony Norcia, Li Fei-Fei, Jiajun Wu
Proceedings of The 7th Conference on Robot Learning, PMLR 229:1737-1760, 2023.

Abstract

We present Neural Signal Operated Intelligent Robots (NOIR), a general-purpose, intelligent brain-robot interface system that enables humans to command robots to perform everyday activities through brain signals. Through this interface, humans communicate their intended objects of interest and actions to the robots using electroencephalography (EEG). Our novel system demonstrates success in an expansive array of 20 challenging, everyday household activities, including cooking, cleaning, personal care, and entertainment. The effectiveness of the system is improved by its synergistic integration of robot learning algorithms, allowing for NOIR to adapt to individual users and predict their intentions. Our work enhances the way humans interact with robots, replacing traditional channels of interaction with direct, neural communication.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-zhang23f, title = {NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities}, author = {Zhang, Ruohan and Lee, Sharon and Hwang, Minjune and Hiranaka, Ayano and Wang, Chen and Ai, Wensi and Tan, Jin Jie Ryan and Gupta, Shreya and Hao, Yilun and Levine, Gabrael and Gao, Ruohan and Norcia, Anthony and Fei-Fei, Li and Wu, Jiajun}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {1737--1760}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/zhang23f/zhang23f.pdf}, url = {https://proceedings.mlr.press/v229/zhang23f.html}, abstract = {We present Neural Signal Operated Intelligent Robots (NOIR), a general-purpose, intelligent brain-robot interface system that enables humans to command robots to perform everyday activities through brain signals. Through this interface, humans communicate their intended objects of interest and actions to the robots using electroencephalography (EEG). Our novel system demonstrates success in an expansive array of 20 challenging, everyday household activities, including cooking, cleaning, personal care, and entertainment. The effectiveness of the system is improved by its synergistic integration of robot learning algorithms, allowing for NOIR to adapt to individual users and predict their intentions. Our work enhances the way humans interact with robots, replacing traditional channels of interaction with direct, neural communication.} }
Endnote
%0 Conference Paper %T NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities %A Ruohan Zhang %A Sharon Lee %A Minjune Hwang %A Ayano Hiranaka %A Chen Wang %A Wensi Ai %A Jin Jie Ryan Tan %A Shreya Gupta %A Yilun Hao %A Gabrael Levine %A Ruohan Gao %A Anthony Norcia %A Li Fei-Fei %A Jiajun Wu %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-zhang23f %I PMLR %P 1737--1760 %U https://proceedings.mlr.press/v229/zhang23f.html %V 229 %X We present Neural Signal Operated Intelligent Robots (NOIR), a general-purpose, intelligent brain-robot interface system that enables humans to command robots to perform everyday activities through brain signals. Through this interface, humans communicate their intended objects of interest and actions to the robots using electroencephalography (EEG). Our novel system demonstrates success in an expansive array of 20 challenging, everyday household activities, including cooking, cleaning, personal care, and entertainment. The effectiveness of the system is improved by its synergistic integration of robot learning algorithms, allowing for NOIR to adapt to individual users and predict their intentions. Our work enhances the way humans interact with robots, replacing traditional channels of interaction with direct, neural communication.
APA
Zhang, R., Lee, S., Hwang, M., Hiranaka, A., Wang, C., Ai, W., Tan, J.J.R., Gupta, S., Hao, Y., Levine, G., Gao, R., Norcia, A., Fei-Fei, L. & Wu, J.. (2023). NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:1737-1760 Available from https://proceedings.mlr.press/v229/zhang23f.html.

Related Material