Stable Coactive Learning via Perturbation

Karthik Raman, Thorsten Joachims, Pannaga Shivaswamy, Tobias Schnabel
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):837-845, 2013.

Abstract

Coactive Learning is a model of interaction between a learning system (e.g. search engine) and its human users, wherein the system learns from (typically implicit) user feedback during operational use. User feedback takes the form of preferences, and recent work has introduced online algorithms that learn from this weak feedback. However, we show that these algorithms can be unstable and ineffective in real-world settings where biases and noise in the feedback are significant. In this paper, we propose the first coactive learning algorithm that can learn robustly despite bias and noise. In particular, we explore how presenting users with slightly perturbed objects (e.g., rankings) can stabilize the learning process. We theoretically validate the algorithm by proving bounds on the average regret. We also provide extensive empirical evidence on benchmarks and from a live search engine user study, showing that the new algorithm substantially outperforms existing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-raman13, title = {Stable Coactive Learning via Perturbation}, author = {Raman, Karthik and Joachims, Thorsten and Shivaswamy, Pannaga and Schnabel, Tobias}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {837--845}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {3}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/raman13.pdf}, url = {https://proceedings.mlr.press/v28/raman13.html}, abstract = {Coactive Learning is a model of interaction between a learning system (e.g. search engine) and its human users, wherein the system learns from (typically implicit) user feedback during operational use. User feedback takes the form of preferences, and recent work has introduced online algorithms that learn from this weak feedback. However, we show that these algorithms can be unstable and ineffective in real-world settings where biases and noise in the feedback are significant. In this paper, we propose the first coactive learning algorithm that can learn robustly despite bias and noise. In particular, we explore how presenting users with slightly perturbed objects (e.g., rankings) can stabilize the learning process. We theoretically validate the algorithm by proving bounds on the average regret. We also provide extensive empirical evidence on benchmarks and from a live search engine user study, showing that the new algorithm substantially outperforms existing methods.} }
Endnote
%0 Conference Paper %T Stable Coactive Learning via Perturbation %A Karthik Raman %A Thorsten Joachims %A Pannaga Shivaswamy %A Tobias Schnabel %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-raman13 %I PMLR %P 837--845 %U https://proceedings.mlr.press/v28/raman13.html %V 28 %N 3 %X Coactive Learning is a model of interaction between a learning system (e.g. search engine) and its human users, wherein the system learns from (typically implicit) user feedback during operational use. User feedback takes the form of preferences, and recent work has introduced online algorithms that learn from this weak feedback. However, we show that these algorithms can be unstable and ineffective in real-world settings where biases and noise in the feedback are significant. In this paper, we propose the first coactive learning algorithm that can learn robustly despite bias and noise. In particular, we explore how presenting users with slightly perturbed objects (e.g., rankings) can stabilize the learning process. We theoretically validate the algorithm by proving bounds on the average regret. We also provide extensive empirical evidence on benchmarks and from a live search engine user study, showing that the new algorithm substantially outperforms existing methods.
RIS
TY - CPAPER TI - Stable Coactive Learning via Perturbation AU - Karthik Raman AU - Thorsten Joachims AU - Pannaga Shivaswamy AU - Tobias Schnabel BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/26 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-raman13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 3 SP - 837 EP - 845 L1 - http://proceedings.mlr.press/v28/raman13.pdf UR - https://proceedings.mlr.press/v28/raman13.html AB - Coactive Learning is a model of interaction between a learning system (e.g. search engine) and its human users, wherein the system learns from (typically implicit) user feedback during operational use. User feedback takes the form of preferences, and recent work has introduced online algorithms that learn from this weak feedback. However, we show that these algorithms can be unstable and ineffective in real-world settings where biases and noise in the feedback are significant. In this paper, we propose the first coactive learning algorithm that can learn robustly despite bias and noise. In particular, we explore how presenting users with slightly perturbed objects (e.g., rankings) can stabilize the learning process. We theoretically validate the algorithm by proving bounds on the average regret. We also provide extensive empirical evidence on benchmarks and from a live search engine user study, showing that the new algorithm substantially outperforms existing methods. ER -
APA
Raman, K., Joachims, T., Shivaswamy, P. & Schnabel, T.. (2013). Stable Coactive Learning via Perturbation. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(3):837-845 Available from https://proceedings.mlr.press/v28/raman13.html.

Related Material