Crowdsourcing with Arbitrary Adversaries

Matthaeus Kleindessner, Pranjal Awasthi
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:2708-2717, 2018.

Abstract

Most existing works on crowdsourcing assume that the workers follow the Dawid-Skene model, or the one-coin model as its special case, where every worker makes mistakes independently of other workers and with the same error probability for every task. We study a significant extension of this restricted model. We allow almost half of the workers to deviate from the one-coin model and for those workers, their probabilities of making an error to be task-dependent and to be arbitrarily correlated. In other words, we allow for arbitrary adversaries, for which not only error probabilities can be high, but which can also perfectly collude. In this adversarial scenario, we design an efficient algorithm to consistently estimate the workers’ error probabilities.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-kleindessner18a, title = {Crowdsourcing with Arbitrary Adversaries}, author = {Kleindessner, Matthaeus and Awasthi, Pranjal}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {2708--2717}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/kleindessner18a/kleindessner18a.pdf}, url = {https://proceedings.mlr.press/v80/kleindessner18a.html}, abstract = {Most existing works on crowdsourcing assume that the workers follow the Dawid-Skene model, or the one-coin model as its special case, where every worker makes mistakes independently of other workers and with the same error probability for every task. We study a significant extension of this restricted model. We allow almost half of the workers to deviate from the one-coin model and for those workers, their probabilities of making an error to be task-dependent and to be arbitrarily correlated. In other words, we allow for arbitrary adversaries, for which not only error probabilities can be high, but which can also perfectly collude. In this adversarial scenario, we design an efficient algorithm to consistently estimate the workers’ error probabilities.} }
Endnote
%0 Conference Paper %T Crowdsourcing with Arbitrary Adversaries %A Matthaeus Kleindessner %A Pranjal Awasthi %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-kleindessner18a %I PMLR %P 2708--2717 %U https://proceedings.mlr.press/v80/kleindessner18a.html %V 80 %X Most existing works on crowdsourcing assume that the workers follow the Dawid-Skene model, or the one-coin model as its special case, where every worker makes mistakes independently of other workers and with the same error probability for every task. We study a significant extension of this restricted model. We allow almost half of the workers to deviate from the one-coin model and for those workers, their probabilities of making an error to be task-dependent and to be arbitrarily correlated. In other words, we allow for arbitrary adversaries, for which not only error probabilities can be high, but which can also perfectly collude. In this adversarial scenario, we design an efficient algorithm to consistently estimate the workers’ error probabilities.
APA
Kleindessner, M. & Awasthi, P.. (2018). Crowdsourcing with Arbitrary Adversaries. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:2708-2717 Available from https://proceedings.mlr.press/v80/kleindessner18a.html.

Related Material