Low-rank extended Kalman filtering for online learning of neural networks from streaming data

Peter G. Chang, Gerardo Durán-Martín, Alex Shestopaloff, Matt Jones, Kevin Patrick Murphy
Proceedings of The 2nd Conference on Lifelong Learning Agents, PMLR 232:1025-1071, 2023.

Abstract

We propose an efficient online approximate Bayesian inference algorithm for estimating the parameters of a nonlinear function from a potentially non-stationary data stream. The method is based on the extended Kalman filter (EKF), but uses a novel low-rank plus diagonal decomposition of the posterior precision matrix, which gives a cost per step which is linear in the number of model parameters. In contrast to methods based on stochastic variational inference, our method is fully deterministic, and does not require step-size tuning. We show experimentally that this results in much faster (more sample efficient) learning, which results in more rapid adaptation to changing distributions, and faster accumulation of reward when used as part of a contextual bandit algorithm.

Cite this Paper

BibTeX
@InProceedings{pmlr-v232-chang23a, title = {Low-rank extended Kalman filtering for online learning of neural networks from streaming data}, author = {Chang, Peter G. and Dur\'an-Mart\'in, Gerardo and Shestopaloff, Alex and Jones, Matt and Murphy, Kevin Patrick}, booktitle = {Proceedings of The 2nd Conference on Lifelong Learning Agents}, pages = {1025--1071}, year = {2023}, editor = {Chandar, Sarath and Pascanu, Razvan and Sedghi, Hanie and Precup, Doina}, volume = {232}, series = {Proceedings of Machine Learning Research}, month = {22--25 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v232/chang23a/chang23a.pdf}, url = {https://proceedings.mlr.press/v232/chang23a.html}, abstract = {We propose an efficient online approximate Bayesian inference algorithm for estimating the parameters of a nonlinear function from a potentially non-stationary data stream. The method is based on the extended Kalman filter (EKF), but uses a novel low-rank plus diagonal decomposition of the posterior precision matrix, which gives a cost per step which is linear in the number of model parameters. In contrast to methods based on stochastic variational inference, our method is fully deterministic, and does not require step-size tuning. We show experimentally that this results in much faster (more sample efficient) learning, which results in more rapid adaptation to changing distributions, and faster accumulation of reward when used as part of a contextual bandit algorithm.} }
Endnote
%0 Conference Paper %T Low-rank extended Kalman filtering for online learning of neural networks from streaming data %A Peter G. Chang %A Gerardo Durán-Martín %A Alex Shestopaloff %A Matt Jones %A Kevin Patrick Murphy %B Proceedings of The 2nd Conference on Lifelong Learning Agents %C Proceedings of Machine Learning Research %D 2023 %E Sarath Chandar %E Razvan Pascanu %E Hanie Sedghi %E Doina Precup %F pmlr-v232-chang23a %I PMLR %P 1025--1071 %U https://proceedings.mlr.press/v232/chang23a.html %V 232 %X We propose an efficient online approximate Bayesian inference algorithm for estimating the parameters of a nonlinear function from a potentially non-stationary data stream. The method is based on the extended Kalman filter (EKF), but uses a novel low-rank plus diagonal decomposition of the posterior precision matrix, which gives a cost per step which is linear in the number of model parameters. In contrast to methods based on stochastic variational inference, our method is fully deterministic, and does not require step-size tuning. We show experimentally that this results in much faster (more sample efficient) learning, which results in more rapid adaptation to changing distributions, and faster accumulation of reward when used as part of a contextual bandit algorithm.
APA
Chang, P.G., Durán-Martín, G., Shestopaloff, A., Jones, M. & Murphy, K.P.. (2023). Low-rank extended Kalman filtering for online learning of neural networks from streaming data. Proceedings of The 2nd Conference on Lifelong Learning Agents, in Proceedings of Machine Learning Research 232:1025-1071 Available from https://proceedings.mlr.press/v232/chang23a.html.

Related Material