Maximum Likelihood Rules for Heterogeneous Voters

Abstract: Aggregating rankings arriving from different sources is a common task both in crowdsourcing (where sources are workers or experts) and in machine learning (where sources may be different algorithms). Under the Condorcet noise model, when all sources are independent and of the same competence, the Kemeny-Young voting rule is known to be the maximum likelihood estimator of the ground truth. When looking for the most likely winner, the answer is more involved but also well understood. We extend all known results on the most likely ranking and winner in the Condorcet noise model to the case where experts have heterogeneous known accuracy levels, by appropriately weighting the experts. We also provide first asymptotic results on the expected distance of some voting rules from the ground truth.