Belief-Grounded Networks for Accelerated Robot Learning under Partial Observability

Hai Nguyen, Brett Daley, Xinchao Song, Christopher Amato, Robert Platt
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:1640-1653, 2021.

Abstract

Many important robotics problems are partially observable where a single visual or force-feedback measurement is insufficient to reconstruct the state. Standard approaches involve learning a policy over beliefs or observation-action histories. However, both of these have drawbacks; it is expensive to track the belief online, and it is hard to learn policies directly over histories. We propose a method for policy learning under partial observability called the Belief-Grounded Network (BGN) in which an auxiliary belief-reconstruction loss incentivizes a neural network to concisely summarize its input history. Since the resulting policy is a function of the history rather than the belief, it can be executed easily at runtime. We compare BGN against several baselines on classic benchmark tasks as well as three novel robotic force-feedback tasks. BGN outperforms all other tested methods and its learned policies work well when transferred onto a physical robot.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-nguyen21a, title = {Belief-Grounded Networks for Accelerated Robot Learning under Partial Observability}, author = {Nguyen, Hai and Daley, Brett and Song, Xinchao and Amato, Christopher and Platt, Robert}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {1640--1653}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/nguyen21a/nguyen21a.pdf}, url = {https://proceedings.mlr.press/v155/nguyen21a.html}, abstract = {Many important robotics problems are partially observable where a single visual or force-feedback measurement is insufficient to reconstruct the state. Standard approaches involve learning a policy over beliefs or observation-action histories. However, both of these have drawbacks; it is expensive to track the belief online, and it is hard to learn policies directly over histories. We propose a method for policy learning under partial observability called the Belief-Grounded Network (BGN) in which an auxiliary belief-reconstruction loss incentivizes a neural network to concisely summarize its input history. Since the resulting policy is a function of the history rather than the belief, it can be executed easily at runtime. We compare BGN against several baselines on classic benchmark tasks as well as three novel robotic force-feedback tasks. BGN outperforms all other tested methods and its learned policies work well when transferred onto a physical robot.} }
Endnote
%0 Conference Paper %T Belief-Grounded Networks for Accelerated Robot Learning under Partial Observability %A Hai Nguyen %A Brett Daley %A Xinchao Song %A Christopher Amato %A Robert Platt %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-nguyen21a %I PMLR %P 1640--1653 %U https://proceedings.mlr.press/v155/nguyen21a.html %V 155 %X Many important robotics problems are partially observable where a single visual or force-feedback measurement is insufficient to reconstruct the state. Standard approaches involve learning a policy over beliefs or observation-action histories. However, both of these have drawbacks; it is expensive to track the belief online, and it is hard to learn policies directly over histories. We propose a method for policy learning under partial observability called the Belief-Grounded Network (BGN) in which an auxiliary belief-reconstruction loss incentivizes a neural network to concisely summarize its input history. Since the resulting policy is a function of the history rather than the belief, it can be executed easily at runtime. We compare BGN against several baselines on classic benchmark tasks as well as three novel robotic force-feedback tasks. BGN outperforms all other tested methods and its learned policies work well when transferred onto a physical robot.
APA
Nguyen, H., Daley, B., Song, X., Amato, C. & Platt, R.. (2021). Belief-Grounded Networks for Accelerated Robot Learning under Partial Observability. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:1640-1653 Available from https://proceedings.mlr.press/v155/nguyen21a.html.

Related Material