Particle-Based Score Estimation for State Space Model Learning in Autonomous Driving

Angad Singh, Omar Makhlouf, Maximilian Igl, Joao Messias, Arnaud Doucet, Shimon Whiteson
Proceedings of The 6th Conference on Robot Learning, PMLR 205:1168-1177, 2023.

Abstract

Multi-object state estimation is a fundamental problem for robotic applications where a robot must interact with other moving objects. Typically, other objects’ relevant state features are not directly observable, and must instead be inferred from observations. Particle filtering can perform such inference given approximate transition and observation models. However, these models are often unknown a priori, yielding a difficult parameter estimation problem since observations jointly carry transition and observation noise. In this work, we consider learning maximum-likelihood parameters using particle methods. Recent methods addressing this problem typically differentiate through time in a particle filter, which requires workarounds to the non-differentiable resampling step, that yield biased or high variance gradient estimates. By contrast, we exploit Fisher’s identity to obtain a particle-based approximation of the score function (the gradient of the log likelihood) that yields a low variance estimate while only requiring stepwise differentiation through the transition and observation models. We apply our method to real data collected from autonomous vehicles (AVs) and show that it learns better models than existing techniques and is more stable in training, yielding an effective smoother for tracking the trajectories of vehicles around an AV.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-singh23a, title = {Particle-Based Score Estimation for State Space Model Learning in Autonomous Driving}, author = {Singh, Angad and Makhlouf, Omar and Igl, Maximilian and Messias, Joao and Doucet, Arnaud and Whiteson, Shimon}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {1168--1177}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/singh23a/singh23a.pdf}, url = {https://proceedings.mlr.press/v205/singh23a.html}, abstract = {Multi-object state estimation is a fundamental problem for robotic applications where a robot must interact with other moving objects. Typically, other objects’ relevant state features are not directly observable, and must instead be inferred from observations. Particle filtering can perform such inference given approximate transition and observation models. However, these models are often unknown a priori, yielding a difficult parameter estimation problem since observations jointly carry transition and observation noise. In this work, we consider learning maximum-likelihood parameters using particle methods. Recent methods addressing this problem typically differentiate through time in a particle filter, which requires workarounds to the non-differentiable resampling step, that yield biased or high variance gradient estimates. By contrast, we exploit Fisher’s identity to obtain a particle-based approximation of the score function (the gradient of the log likelihood) that yields a low variance estimate while only requiring stepwise differentiation through the transition and observation models. We apply our method to real data collected from autonomous vehicles (AVs) and show that it learns better models than existing techniques and is more stable in training, yielding an effective smoother for tracking the trajectories of vehicles around an AV.} }
Endnote
%0 Conference Paper %T Particle-Based Score Estimation for State Space Model Learning in Autonomous Driving %A Angad Singh %A Omar Makhlouf %A Maximilian Igl %A Joao Messias %A Arnaud Doucet %A Shimon Whiteson %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-singh23a %I PMLR %P 1168--1177 %U https://proceedings.mlr.press/v205/singh23a.html %V 205 %X Multi-object state estimation is a fundamental problem for robotic applications where a robot must interact with other moving objects. Typically, other objects’ relevant state features are not directly observable, and must instead be inferred from observations. Particle filtering can perform such inference given approximate transition and observation models. However, these models are often unknown a priori, yielding a difficult parameter estimation problem since observations jointly carry transition and observation noise. In this work, we consider learning maximum-likelihood parameters using particle methods. Recent methods addressing this problem typically differentiate through time in a particle filter, which requires workarounds to the non-differentiable resampling step, that yield biased or high variance gradient estimates. By contrast, we exploit Fisher’s identity to obtain a particle-based approximation of the score function (the gradient of the log likelihood) that yields a low variance estimate while only requiring stepwise differentiation through the transition and observation models. We apply our method to real data collected from autonomous vehicles (AVs) and show that it learns better models than existing techniques and is more stable in training, yielding an effective smoother for tracking the trajectories of vehicles around an AV.
APA
Singh, A., Makhlouf, O., Igl, M., Messias, J., Doucet, A. & Whiteson, S.. (2023). Particle-Based Score Estimation for State Space Model Learning in Autonomous Driving. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:1168-1177 Available from https://proceedings.mlr.press/v205/singh23a.html.

Related Material