Empirical bayes approach to truth discovery problems

Tsviel Ben Shabat, Reshef Meir, David Azriel
Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, PMLR 180:150-158, 2022.

Abstract

When aggregating information from conflicting sources, one’s goal is to find the truth. Most real-value truth discovery (TD) algorithms try to achieve this goal by estimating the competence of each source and then aggregating the conflicting information by weighing each source’s answer proportionally to her competence. However, each of those algorithms requires more than a single source for such estimation and usually does not consider different estimation methods other than a weighted mean. Therefore, in this work we formulate, prove, and empirically test the conditions for an Empirical Bayes Estimator (EBE) to dominate the weighted mean aggregation. Our main result demonstrates that EBE, under mild conditions, can be used as a second step of any TD algorithm in order to reduce the expected error.

Cite this Paper

BibTeX
@InProceedings{pmlr-v180-ben-shabat22a, title = {Empirical bayes approach to truth discovery problems}, author = {Ben Shabat, Tsviel and Meir, Reshef and Azriel, David}, booktitle = {Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence}, pages = {150--158}, year = {2022}, editor = {Cussens, James and Zhang, Kun}, volume = {180}, series = {Proceedings of Machine Learning Research}, month = {01--05 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v180/ben-shabat22a/ben-shabat22a.pdf}, url = {https://proceedings.mlr.press/v180/ben-shabat22a.html}, abstract = {When aggregating information from conflicting sources, one’s goal is to find the truth. Most real-value truth discovery (TD) algorithms try to achieve this goal by estimating the competence of each source and then aggregating the conflicting information by weighing each source’s answer proportionally to her competence. However, each of those algorithms requires more than a single source for such estimation and usually does not consider different estimation methods other than a weighted mean. Therefore, in this work we formulate, prove, and empirically test the conditions for an Empirical Bayes Estimator (EBE) to dominate the weighted mean aggregation. Our main result demonstrates that EBE, under mild conditions, can be used as a second step of any TD algorithm in order to reduce the expected error.} }
Endnote
%0 Conference Paper %T Empirical bayes approach to truth discovery problems %A Tsviel Ben Shabat %A Reshef Meir %A David Azriel %B Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2022 %E James Cussens %E Kun Zhang %F pmlr-v180-ben-shabat22a %I PMLR %P 150--158 %U https://proceedings.mlr.press/v180/ben-shabat22a.html %V 180 %X When aggregating information from conflicting sources, one’s goal is to find the truth. Most real-value truth discovery (TD) algorithms try to achieve this goal by estimating the competence of each source and then aggregating the conflicting information by weighing each source’s answer proportionally to her competence. However, each of those algorithms requires more than a single source for such estimation and usually does not consider different estimation methods other than a weighted mean. Therefore, in this work we formulate, prove, and empirically test the conditions for an Empirical Bayes Estimator (EBE) to dominate the weighted mean aggregation. Our main result demonstrates that EBE, under mild conditions, can be used as a second step of any TD algorithm in order to reduce the expected error.
APA
Ben Shabat, T., Meir, R. & Azriel, D.. (2022). Empirical bayes approach to truth discovery problems. Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 180:150-158 Available from https://proceedings.mlr.press/v180/ben-shabat22a.html.

Related Material