PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review

[edit]

Ivan Stelmakh, Nihar B. Shah, Aarti Singh ;
Proceedings of the 30th International Conference on Algorithmic Learning Theory, PMLR 98:828-856, 2019.

Abstract

We consider the problem of automated assignment of papers to reviewers in conference peer review, with a focus on fairness and statistical accuracy. Our fairness objective is to maximize the review quality of the most disadvantaged paper, in contrast to the popular objective of maximizing the total quality over all papers. We design an assignment algorithm based on an incremental max-flow procedure that we prove is near-optimally fair. Our statistical accuracy objective is to ensure correct recovery of the papers that should be accepted. With a sharp minimax analysis we also prove that our algorithm leads to assignments with strong statistical guarantees both in an objective-score model as well as a novel subjective-score model that we propose in this paper.

Related Material