Racing Thompson: an Efficient Algorithm for Thompson Sampling with Nonconjugate Priors
[edit]
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:60006008, 2018.
Abstract
Thompson sampling has impressive empirical performance for many multiarmed bandit problems. But current algorithms for Thompson sampling only work for the case of conjugate priors since they require to perform online Bayesian posterior inference, which is a difficult task when the prior is not conjugate. In this paper, we propose a novel algorithm for Thompson sampling which only requires to draw samples from a tractable proposal distribution. So our algorithm is efficient even when the prior is nonconjugate. To do this, we reformulate Thompson sampling as an optimization proplem via the GumbelMax trick. After that we construct a set of random variables and our goal is to identify the one with highest mean which is an instance of best arm identification problems. Finally, we solve it with techniques in best arm identification. Experiments show that our algorithm works well in practice.
Related Material


