A Cost-Effective Framework for Preference Elicitation and Aggregation

Zhibing Zhao, Haoming Li, Junming Wang, Jeffrey O. Kephart, Nicholas Mattei, Hui Su, Lirong Xia
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:445-455, 2018.

Abstract

We propose a cost-effective framework for preference elicitation and aggregation under the Plackett-Luce model with features. Given a budget, our framework iteratively computes the most cost-effective elicitation questions in order to help the agents make a better group decision. We illustrate the viability of the framework with experiments on Amazon Mechanical Turk, which we use to estimate the cost of answering different types of elicitation ques- tions. We compare the prediction accuracy of our framework when adopting various infor- mation criteria that evaluate the expected infor- mation gain from a question. Our experiments show carefully designed information criteria are much more efficient, i.e., they arrive at the correct answer using fewer queries, than ran- domly asking questions given the budget con- straint.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-zhao18b, title = {A Cost-Effective Framework for Preference Elicitation and Aggregation}, author = {Zhao, Zhibing and Li, Haoming and Wang, Junming and Kephart, Jeffrey O. and Mattei, Nicholas and Su, Hui and Xia, Lirong}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {445--455}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/zhao18b/zhao18b.pdf}, url = {https://proceedings.mlr.press/r16/zhao18b.html}, abstract = {We propose a cost-effective framework for preference elicitation and aggregation under the Plackett-Luce model with features. Given a budget, our framework iteratively computes the most cost-effective elicitation questions in order to help the agents make a better group decision. We illustrate the viability of the framework with experiments on Amazon Mechanical Turk, which we use to estimate the cost of answering different types of elicitation ques- tions. We compare the prediction accuracy of our framework when adopting various infor- mation criteria that evaluate the expected infor- mation gain from a question. Our experiments show carefully designed information criteria are much more efficient, i.e., they arrive at the correct answer using fewer queries, than ran- domly asking questions given the budget con- straint.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T A Cost-Effective Framework for Preference Elicitation and Aggregation %A Zhibing Zhao %A Haoming Li %A Junming Wang %A Jeffrey O. Kephart %A Nicholas Mattei %A Hui Su %A Lirong Xia %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-zhao18b %I PMLR %P 445--455 %U https://proceedings.mlr.press/r16/zhao18b.html %V R16 %X We propose a cost-effective framework for preference elicitation and aggregation under the Plackett-Luce model with features. Given a budget, our framework iteratively computes the most cost-effective elicitation questions in order to help the agents make a better group decision. We illustrate the viability of the framework with experiments on Amazon Mechanical Turk, which we use to estimate the cost of answering different types of elicitation ques- tions. We compare the prediction accuracy of our framework when adopting various infor- mation criteria that evaluate the expected infor- mation gain from a question. Our experiments show carefully designed information criteria are much more efficient, i.e., they arrive at the correct answer using fewer queries, than ran- domly asking questions given the budget con- straint. %Z Reissued by PMLR on 04 October 2026.
APA
Zhao, Z., Li, H., Wang, J., Kephart, J.O., Mattei, N., Su, H. & Xia, L.. (2018). A Cost-Effective Framework for Preference Elicitation and Aggregation. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:445-455 Available from https://proceedings.mlr.press/r16/zhao18b.html. Reissued by PMLR on 04 October 2026.

Related Material