BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment

Zeyi Liu, Zhenjia Xu, Shuran Song
Proceedings of The 6th Conference on Robot Learning, PMLR 205:505-515, 2023.

Abstract

We introduce BusyBoard, a toy-inspired robot learning environment that leverages a diverse set of articulated objects and inter-object functional relations to provide rich visual feedback for robot interactions. Based on this environment, we introduce a learning framework, BusyBot, which allows an agent to jointly acquire three fundamental capabilities (interaction, reasoning, and planning) in an integrated and self-supervised manner. With the rich sensory feedback provided by BusyBoard, BusyBot first learns a policy to efficiently interact with the environment; then with data collected using the policy, BusyBot reasons the inter-object functional relations through a causal discovery network; and finally by combining the learned interaction policy and relation reasoning skill, the agent is able to perform goal-conditioned manipulation tasks. We evaluate BusyBot in both simulated and real-world environments, and validate its generalizability to unseen objects and relations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-liu23c, title = {BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment}, author = {Liu, Zeyi and Xu, Zhenjia and Song, Shuran}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {505--515}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/liu23c/liu23c.pdf}, url = {https://proceedings.mlr.press/v205/liu23c.html}, abstract = {We introduce BusyBoard, a toy-inspired robot learning environment that leverages a diverse set of articulated objects and inter-object functional relations to provide rich visual feedback for robot interactions. Based on this environment, we introduce a learning framework, BusyBot, which allows an agent to jointly acquire three fundamental capabilities (interaction, reasoning, and planning) in an integrated and self-supervised manner. With the rich sensory feedback provided by BusyBoard, BusyBot first learns a policy to efficiently interact with the environment; then with data collected using the policy, BusyBot reasons the inter-object functional relations through a causal discovery network; and finally by combining the learned interaction policy and relation reasoning skill, the agent is able to perform goal-conditioned manipulation tasks. We evaluate BusyBot in both simulated and real-world environments, and validate its generalizability to unseen objects and relations.} }
Endnote
%0 Conference Paper %T BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment %A Zeyi Liu %A Zhenjia Xu %A Shuran Song %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-liu23c %I PMLR %P 505--515 %U https://proceedings.mlr.press/v205/liu23c.html %V 205 %X We introduce BusyBoard, a toy-inspired robot learning environment that leverages a diverse set of articulated objects and inter-object functional relations to provide rich visual feedback for robot interactions. Based on this environment, we introduce a learning framework, BusyBot, which allows an agent to jointly acquire three fundamental capabilities (interaction, reasoning, and planning) in an integrated and self-supervised manner. With the rich sensory feedback provided by BusyBoard, BusyBot first learns a policy to efficiently interact with the environment; then with data collected using the policy, BusyBot reasons the inter-object functional relations through a causal discovery network; and finally by combining the learned interaction policy and relation reasoning skill, the agent is able to perform goal-conditioned manipulation tasks. We evaluate BusyBot in both simulated and real-world environments, and validate its generalizability to unseen objects and relations.
APA
Liu, Z., Xu, Z. & Song, S.. (2023). BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:505-515 Available from https://proceedings.mlr.press/v205/liu23c.html.

Related Material