Memory-Based Meta-Learning on Non-Stationary Distributions

Tim Genewein, Gregoire Deletang, Anian Ruoss, Li Kevin Wenliang, Elliot Catt, Vincent Dutordoir, Jordi Grau-Moya, Laurent Orseau, Marcus Hutter, Joel Veness
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:11173-11195, 2023.

Abstract

Memory-based meta-learning is a technique for approximating Bayes-optimal predictors. Under fairly general conditions, minimizing sequential prediction error, measured by the log loss, leads to implicit meta-learning. The goal of this work is to investigate how far this interpretation can be realized by current sequence prediction models and training regimes. The focus is on piecewise stationary sources with unobserved switching-points, which arguably capture an important characteristic of natural language and action-observation sequences in partially observable environments. We show that various types of memory-based neural models, including Transformers, LSTMs, and RNNs can learn to accurately approximate known Bayes-optimal algorithms and behave as if performing Bayesian inference over the latent switching-points and the latent parameters governing the data distribution within each segment.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-genewein23a, title = {Memory-Based Meta-Learning on Non-Stationary Distributions}, author = {Genewein, Tim and Deletang, Gregoire and Ruoss, Anian and Wenliang, Li Kevin and Catt, Elliot and Dutordoir, Vincent and Grau-Moya, Jordi and Orseau, Laurent and Hutter, Marcus and Veness, Joel}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {11173--11195}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/genewein23a/genewein23a.pdf}, url = {https://proceedings.mlr.press/v202/genewein23a.html}, abstract = {Memory-based meta-learning is a technique for approximating Bayes-optimal predictors. Under fairly general conditions, minimizing sequential prediction error, measured by the log loss, leads to implicit meta-learning. The goal of this work is to investigate how far this interpretation can be realized by current sequence prediction models and training regimes. The focus is on piecewise stationary sources with unobserved switching-points, which arguably capture an important characteristic of natural language and action-observation sequences in partially observable environments. We show that various types of memory-based neural models, including Transformers, LSTMs, and RNNs can learn to accurately approximate known Bayes-optimal algorithms and behave as if performing Bayesian inference over the latent switching-points and the latent parameters governing the data distribution within each segment.} }
Endnote
%0 Conference Paper %T Memory-Based Meta-Learning on Non-Stationary Distributions %A Tim Genewein %A Gregoire Deletang %A Anian Ruoss %A Li Kevin Wenliang %A Elliot Catt %A Vincent Dutordoir %A Jordi Grau-Moya %A Laurent Orseau %A Marcus Hutter %A Joel Veness %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-genewein23a %I PMLR %P 11173--11195 %U https://proceedings.mlr.press/v202/genewein23a.html %V 202 %X Memory-based meta-learning is a technique for approximating Bayes-optimal predictors. Under fairly general conditions, minimizing sequential prediction error, measured by the log loss, leads to implicit meta-learning. The goal of this work is to investigate how far this interpretation can be realized by current sequence prediction models and training regimes. The focus is on piecewise stationary sources with unobserved switching-points, which arguably capture an important characteristic of natural language and action-observation sequences in partially observable environments. We show that various types of memory-based neural models, including Transformers, LSTMs, and RNNs can learn to accurately approximate known Bayes-optimal algorithms and behave as if performing Bayesian inference over the latent switching-points and the latent parameters governing the data distribution within each segment.
APA
Genewein, T., Deletang, G., Ruoss, A., Wenliang, L.K., Catt, E., Dutordoir, V., Grau-Moya, J., Orseau, L., Hutter, M. & Veness, J.. (2023). Memory-Based Meta-Learning on Non-Stationary Distributions. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:11173-11195 Available from https://proceedings.mlr.press/v202/genewein23a.html.

Related Material