On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm

Julien Aubert, Luc Lehéricy, Patricia Reynaud-Bouret
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:1244-1275, 2023.

Abstract

When fitting the learning data of an individual to algorithm-like learning models, the observations are so dependent and non-stationary that one may wonder what the classical Maximum Likelihood Estimator (MLE) could do, even if it is the usual tool applied to experimental cognition. Our objective in this work is to show that the estimation of the learning rate cannot be efficient if the learning rate is constant in the classical Exp3 (Exponential weights for Exploration and Exploitation) algorithm. Secondly, we show that if the learning rate decreases polynomially with the sample size, then the prediction error and in some cases the estimation error of the MLE satisfy bounds in probability that decrease at a polynomial rate.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-aubert23a, title = {On the convergence of the {MLE} as an estimator of the learning rate in the Exp3 algorithm}, author = {Aubert, Julien and Leh\'{e}ricy, Luc and Reynaud-Bouret, Patricia}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {1244--1275}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/aubert23a/aubert23a.pdf}, url = {https://proceedings.mlr.press/v202/aubert23a.html}, abstract = {When fitting the learning data of an individual to algorithm-like learning models, the observations are so dependent and non-stationary that one may wonder what the classical Maximum Likelihood Estimator (MLE) could do, even if it is the usual tool applied to experimental cognition. Our objective in this work is to show that the estimation of the learning rate cannot be efficient if the learning rate is constant in the classical Exp3 (Exponential weights for Exploration and Exploitation) algorithm. Secondly, we show that if the learning rate decreases polynomially with the sample size, then the prediction error and in some cases the estimation error of the MLE satisfy bounds in probability that decrease at a polynomial rate.} }
Endnote
%0 Conference Paper %T On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm %A Julien Aubert %A Luc Lehéricy %A Patricia Reynaud-Bouret %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-aubert23a %I PMLR %P 1244--1275 %U https://proceedings.mlr.press/v202/aubert23a.html %V 202 %X When fitting the learning data of an individual to algorithm-like learning models, the observations are so dependent and non-stationary that one may wonder what the classical Maximum Likelihood Estimator (MLE) could do, even if it is the usual tool applied to experimental cognition. Our objective in this work is to show that the estimation of the learning rate cannot be efficient if the learning rate is constant in the classical Exp3 (Exponential weights for Exploration and Exploitation) algorithm. Secondly, we show that if the learning rate decreases polynomially with the sample size, then the prediction error and in some cases the estimation error of the MLE satisfy bounds in probability that decrease at a polynomial rate.
APA
Aubert, J., Lehéricy, L. & Reynaud-Bouret, P.. (2023). On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:1244-1275 Available from https://proceedings.mlr.press/v202/aubert23a.html.

Related Material