Reducing Crowdsourcing to Graphon Estimation, Statistically

Devavrat Shah, Christina Lee
Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, PMLR 84:1741-1750, 2018.

Abstract

Inferring the correct answers to binary tasks based on multiple noisy answers in an unsupervised manner has emerged as the canonical question for micro-task crowdsourcing or more generally aggregating opinions. In graphon estimation, one is interested in estimating edge intensities or probabilities between nodes using a single snapshot of a graph realization. In the recent literature, there has been exciting development within both of these topics. In the context of crowdsourcing, the key intellectual challenge is to understand whether a given task can be more accurately denoised by aggregating answers collected from other different tasks. In the context of graphon estimation, precise information limits and estimation algorithms remain of interest. In this paper, we utilize a statistical reduction from crowdsourcing to graphon estimation to advance the state-of-art for both of these challenges. We use concepts from graphon estimation to design an algorithm that achieves better performance than the majority voting scheme for a setup that goes beyond the rank one models considered in the literature. We use known lower bounds for crowdsourcing to derive lower bounds for graphon estimation.

Cite this Paper


BibTeX
@InProceedings{pmlr-v84-shah18a, title = {Reducing Crowdsourcing to Graphon Estimation, Statistically}, author = {Shah, Devavrat and Lee, Christina}, booktitle = {Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics}, pages = {1741--1750}, year = {2018}, editor = {Storkey, Amos and Perez-Cruz, Fernando}, volume = {84}, series = {Proceedings of Machine Learning Research}, month = {09--11 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v84/shah18a/shah18a.pdf}, url = {https://proceedings.mlr.press/v84/shah18a.html}, abstract = {Inferring the correct answers to binary tasks based on multiple noisy answers in an unsupervised manner has emerged as the canonical question for micro-task crowdsourcing or more generally aggregating opinions. In graphon estimation, one is interested in estimating edge intensities or probabilities between nodes using a single snapshot of a graph realization. In the recent literature, there has been exciting development within both of these topics. In the context of crowdsourcing, the key intellectual challenge is to understand whether a given task can be more accurately denoised by aggregating answers collected from other different tasks. In the context of graphon estimation, precise information limits and estimation algorithms remain of interest. In this paper, we utilize a statistical reduction from crowdsourcing to graphon estimation to advance the state-of-art for both of these challenges. We use concepts from graphon estimation to design an algorithm that achieves better performance than the majority voting scheme for a setup that goes beyond the rank one models considered in the literature. We use known lower bounds for crowdsourcing to derive lower bounds for graphon estimation. } }
Endnote
%0 Conference Paper %T Reducing Crowdsourcing to Graphon Estimation, Statistically %A Devavrat Shah %A Christina Lee %B Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2018 %E Amos Storkey %E Fernando Perez-Cruz %F pmlr-v84-shah18a %I PMLR %P 1741--1750 %U https://proceedings.mlr.press/v84/shah18a.html %V 84 %X Inferring the correct answers to binary tasks based on multiple noisy answers in an unsupervised manner has emerged as the canonical question for micro-task crowdsourcing or more generally aggregating opinions. In graphon estimation, one is interested in estimating edge intensities or probabilities between nodes using a single snapshot of a graph realization. In the recent literature, there has been exciting development within both of these topics. In the context of crowdsourcing, the key intellectual challenge is to understand whether a given task can be more accurately denoised by aggregating answers collected from other different tasks. In the context of graphon estimation, precise information limits and estimation algorithms remain of interest. In this paper, we utilize a statistical reduction from crowdsourcing to graphon estimation to advance the state-of-art for both of these challenges. We use concepts from graphon estimation to design an algorithm that achieves better performance than the majority voting scheme for a setup that goes beyond the rank one models considered in the literature. We use known lower bounds for crowdsourcing to derive lower bounds for graphon estimation.
APA
Shah, D. & Lee, C.. (2018). Reducing Crowdsourcing to Graphon Estimation, Statistically. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 84:1741-1750 Available from https://proceedings.mlr.press/v84/shah18a.html.

Related Material