A Deep Learning Approach to Unsupervised Ensemble Learning

Uri Shaham, Xiuyuan Cheng, Omer Dror, Ariel Jaffe, Boaz Nadler, Joseph Chang, Yuval Kluger
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:30-39, 2016.

Abstract

We show how deep learning methods can be applied in the context of crowdsourcing and unsupervised ensemble learning. First, we prove that the popular model of Dawid and Skene, which assumes that all classifiers are conditionally independent, is \em equivalent to a Restricted Boltzmann Machine (RBM) with a single hidden node. Hence, under this model, the posterior probabilities of the true labels can be instead estimated via a trained RBM. Next, to address the more general case, where classifiers may strongly violate the conditional independence assumption, we propose to apply RBM-based Deep Neural Net (DNN). Experimental results on various simulated and real-world datasets demonstrate that our proposed DNN approach outperforms other state-of-the-art methods, in particular when the data violates the conditional independence assumption.

Cite this Paper

BibTeX
@InProceedings{pmlr-v48-shaham16, title = {A Deep Learning Approach to Unsupervised Ensemble Learning}, author = {Shaham, Uri and Cheng, Xiuyuan and Dror, Omer and Jaffe, Ariel and Nadler, Boaz and Chang, Joseph and Kluger, Yuval}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {30--39}, year = {2016}, editor = {Balcan, Maria Florina and Weinberger, Kilian Q.}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/shaham16.pdf}, url = {https://proceedings.mlr.press/v48/shaham16.html}, abstract = {We show how deep learning methods can be applied in the context of crowdsourcing and unsupervised ensemble learning. First, we prove that the popular model of Dawid and Skene, which assumes that all classifiers are conditionally independent, is \em equivalent to a Restricted Boltzmann Machine (RBM) with a single hidden node. Hence, under this model, the posterior probabilities of the true labels can be instead estimated via a trained RBM. Next, to address the more general case, where classifiers may strongly violate the conditional independence assumption, we propose to apply RBM-based Deep Neural Net (DNN). Experimental results on various simulated and real-world datasets demonstrate that our proposed DNN approach outperforms other state-of-the-art methods, in particular when the data violates the conditional independence assumption.} }
Endnote
%0 Conference Paper %T A Deep Learning Approach to Unsupervised Ensemble Learning %A Uri Shaham %A Xiuyuan Cheng %A Omer Dror %A Ariel Jaffe %A Boaz Nadler %A Joseph Chang %A Yuval Kluger %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-shaham16 %I PMLR %P 30--39 %U https://proceedings.mlr.press/v48/shaham16.html %V 48 %X We show how deep learning methods can be applied in the context of crowdsourcing and unsupervised ensemble learning. First, we prove that the popular model of Dawid and Skene, which assumes that all classifiers are conditionally independent, is \em equivalent to a Restricted Boltzmann Machine (RBM) with a single hidden node. Hence, under this model, the posterior probabilities of the true labels can be instead estimated via a trained RBM. Next, to address the more general case, where classifiers may strongly violate the conditional independence assumption, we propose to apply RBM-based Deep Neural Net (DNN). Experimental results on various simulated and real-world datasets demonstrate that our proposed DNN approach outperforms other state-of-the-art methods, in particular when the data violates the conditional independence assumption.
RIS
TY - CPAPER TI - A Deep Learning Approach to Unsupervised Ensemble Learning AU - Uri Shaham AU - Xiuyuan Cheng AU - Omer Dror AU - Ariel Jaffe AU - Boaz Nadler AU - Joseph Chang AU - Yuval Kluger BT - Proceedings of The 33rd International Conference on Machine Learning DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-shaham16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 48 SP - 30 EP - 39 L1 - http://proceedings.mlr.press/v48/shaham16.pdf UR - https://proceedings.mlr.press/v48/shaham16.html AB - We show how deep learning methods can be applied in the context of crowdsourcing and unsupervised ensemble learning. First, we prove that the popular model of Dawid and Skene, which assumes that all classifiers are conditionally independent, is \em equivalent to a Restricted Boltzmann Machine (RBM) with a single hidden node. Hence, under this model, the posterior probabilities of the true labels can be instead estimated via a trained RBM. Next, to address the more general case, where classifiers may strongly violate the conditional independence assumption, we propose to apply RBM-based Deep Neural Net (DNN). Experimental results on various simulated and real-world datasets demonstrate that our proposed DNN approach outperforms other state-of-the-art methods, in particular when the data violates the conditional independence assumption. ER -
APA
Shaham, U., Cheng, X., Dror, O., Jaffe, A., Nadler, B., Chang, J. & Kluger, Y.. (2016). A Deep Learning Approach to Unsupervised Ensemble Learning. Proceedings of The 33rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 48:30-39 Available from https://proceedings.mlr.press/v48/shaham16.html.

Related Material