Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback

Chicheng Zhang, Alekh Agarwal, Hal Daumé Iii, John Langford, Sahand Negahban
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:7335-7344, 2019.

Abstract

We investigate the feasibility of learning from both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to divergences between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approaches are feasible, and helpful in practice.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-zhang19b, title = {Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback}, author = {Zhang, Chicheng and Agarwal, Alekh and Iii, Hal Daum{\'e} and Langford, John and Negahban, Sahand}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {7335--7344}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/zhang19b/zhang19b.pdf}, url = {https://proceedings.mlr.press/v97/zhang19b.html}, abstract = {We investigate the feasibility of learning from both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to divergences between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approaches are feasible, and helpful in practice.} }
Endnote
%0 Conference Paper %T Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback %A Chicheng Zhang %A Alekh Agarwal %A Hal Daumé Iii %A John Langford %A Sahand Negahban %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-zhang19b %I PMLR %P 7335--7344 %U https://proceedings.mlr.press/v97/zhang19b.html %V 97 %X We investigate the feasibility of learning from both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to divergences between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approaches are feasible, and helpful in practice.
APA
Zhang, C., Agarwal, A., Iii, H.D., Langford, J. & Negahban, S.. (2019). Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:7335-7344 Available from https://proceedings.mlr.press/v97/zhang19b.html.

Related Material