STReSSD: Sim-To-Real from Sound for Stochastic Dynamics

Carolyn Matl, Yashraj Narang, Dieter Fox, Ruzena Bajcsy, Fabio Ramos
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:935-958, 2021.

Abstract

Sound is an information-rich medium that captures dynamic physical events. This work presents STReSSD, a framework that uses sound to bridge the simulation-to-reality gap for stochastic dynamics, demonstrated for the canonical case of a bouncing ball. A physically-motivated noise model is presented to capture stochastic behavior of the balls upon collision with the environment. A likelihood-free Bayesian inference framework is used to infer the parameters of the noise model, as well as a material property called the coefficient of restitution, from audio observations. The same inference framework and the calibrated stochastic simulator are then used to learn a probabilistic model of ball dynamics. The predictive capabilities of the dynamics model are tested in two robotic experiments. First, open-loop predictions anticipate probabilistic success of bouncing a ball into a cup. The second experiment integrates audio perception with a robotic arm to track and deflect a bouncing ball in real-time. We envision that this work is a step towards integrating audio-based inference for dynamic robotic tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-matl21a, title = {STReSSD: Sim-To-Real from Sound for Stochastic Dynamics}, author = {Matl, Carolyn and Narang, Yashraj and Fox, Dieter and Bajcsy, Ruzena and Ramos, Fabio}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {935--958}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/matl21a/matl21a.pdf}, url = {https://proceedings.mlr.press/v155/matl21a.html}, abstract = {Sound is an information-rich medium that captures dynamic physical events. This work presents STReSSD, a framework that uses sound to bridge the simulation-to-reality gap for stochastic dynamics, demonstrated for the canonical case of a bouncing ball. A physically-motivated noise model is presented to capture stochastic behavior of the balls upon collision with the environment. A likelihood-free Bayesian inference framework is used to infer the parameters of the noise model, as well as a material property called the coefficient of restitution, from audio observations. The same inference framework and the calibrated stochastic simulator are then used to learn a probabilistic model of ball dynamics. The predictive capabilities of the dynamics model are tested in two robotic experiments. First, open-loop predictions anticipate probabilistic success of bouncing a ball into a cup. The second experiment integrates audio perception with a robotic arm to track and deflect a bouncing ball in real-time. We envision that this work is a step towards integrating audio-based inference for dynamic robotic tasks.} }
Endnote
%0 Conference Paper %T STReSSD: Sim-To-Real from Sound for Stochastic Dynamics %A Carolyn Matl %A Yashraj Narang %A Dieter Fox %A Ruzena Bajcsy %A Fabio Ramos %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-matl21a %I PMLR %P 935--958 %U https://proceedings.mlr.press/v155/matl21a.html %V 155 %X Sound is an information-rich medium that captures dynamic physical events. This work presents STReSSD, a framework that uses sound to bridge the simulation-to-reality gap for stochastic dynamics, demonstrated for the canonical case of a bouncing ball. A physically-motivated noise model is presented to capture stochastic behavior of the balls upon collision with the environment. A likelihood-free Bayesian inference framework is used to infer the parameters of the noise model, as well as a material property called the coefficient of restitution, from audio observations. The same inference framework and the calibrated stochastic simulator are then used to learn a probabilistic model of ball dynamics. The predictive capabilities of the dynamics model are tested in two robotic experiments. First, open-loop predictions anticipate probabilistic success of bouncing a ball into a cup. The second experiment integrates audio perception with a robotic arm to track and deflect a bouncing ball in real-time. We envision that this work is a step towards integrating audio-based inference for dynamic robotic tasks.
APA
Matl, C., Narang, Y., Fox, D., Bajcsy, R. & Ramos, F.. (2021). STReSSD: Sim-To-Real from Sound for Stochastic Dynamics. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:935-958 Available from https://proceedings.mlr.press/v155/matl21a.html.

Related Material