Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes

Jongseok Lee, Jianxiang Feng, Matthias Humt, Marcus Gerhard Müller, Rudolph Triebel
Proceedings of the 5th Conference on Robot Learning, PMLR 164:1168-1179, 2022.

Abstract

This paper presents a probabilistic framework to obtain both reliable and fast uncertainty estimates for predictions with Deep Neural Networks (DNNs). Our main contribution is a practical and principled combination of DNNs with sparse Gaussian Processes (GPs). We prove theoretically that DNNs can be seen as a special case of sparse GPs, namely mixtures of GP experts (MoE-GP), and we devise a learning algorithm that brings the derived theory into practice. In experiments from two different robotic tasks – inverse dynamics of a manipulator and object detection on a micro-aerial vehicle (MAV) – we show the effectiveness of our approach in terms of predictive uncertainty, improved scalability, and run-time efficiency on a Jetson TX2. We thus argue that our approach can pave the way towards reliable and fast robot learning systems with uncertainty awareness.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-lee22c, title = {Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes}, author = {Lee, Jongseok and Feng, Jianxiang and Humt, Matthias and M\"uller, Marcus Gerhard and Triebel, Rudolph}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {1168--1179}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/lee22c/lee22c.pdf}, url = {https://proceedings.mlr.press/v164/lee22c.html}, abstract = {This paper presents a probabilistic framework to obtain both reliable and fast uncertainty estimates for predictions with Deep Neural Networks (DNNs). Our main contribution is a practical and principled combination of DNNs with sparse Gaussian Processes (GPs). We prove theoretically that DNNs can be seen as a special case of sparse GPs, namely mixtures of GP experts (MoE-GP), and we devise a learning algorithm that brings the derived theory into practice. In experiments from two different robotic tasks – inverse dynamics of a manipulator and object detection on a micro-aerial vehicle (MAV) – we show the effectiveness of our approach in terms of predictive uncertainty, improved scalability, and run-time efficiency on a Jetson TX2. We thus argue that our approach can pave the way towards reliable and fast robot learning systems with uncertainty awareness.} }
Endnote
%0 Conference Paper %T Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes %A Jongseok Lee %A Jianxiang Feng %A Matthias Humt %A Marcus Gerhard Müller %A Rudolph Triebel %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-lee22c %I PMLR %P 1168--1179 %U https://proceedings.mlr.press/v164/lee22c.html %V 164 %X This paper presents a probabilistic framework to obtain both reliable and fast uncertainty estimates for predictions with Deep Neural Networks (DNNs). Our main contribution is a practical and principled combination of DNNs with sparse Gaussian Processes (GPs). We prove theoretically that DNNs can be seen as a special case of sparse GPs, namely mixtures of GP experts (MoE-GP), and we devise a learning algorithm that brings the derived theory into practice. In experiments from two different robotic tasks – inverse dynamics of a manipulator and object detection on a micro-aerial vehicle (MAV) – we show the effectiveness of our approach in terms of predictive uncertainty, improved scalability, and run-time efficiency on a Jetson TX2. We thus argue that our approach can pave the way towards reliable and fast robot learning systems with uncertainty awareness.
APA
Lee, J., Feng, J., Humt, M., Müller, M.G. & Triebel, R.. (2022). Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:1168-1179 Available from https://proceedings.mlr.press/v164/lee22c.html.

Related Material