Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback


Chicheng Zhang, Alekh Agarwal, Hal Daumé Iii, John Langford, Sahand Negahban ;
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:7335-7344, 2019.


We investigate the feasibility of learning from both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to divergences between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approaches are feasible, and helpful in practice.

Related Material