We propose a cost-effective framework for preference elicitation and aggregation under the Plackett-Luce model with features. Given a budget, our framework iteratively computes the most cost-effective elicitation questions in order to help the agents make a better group decision. We illustrate the viability of the framework with experiments on Amazon Mechanical Turk, which we use to estimate the cost of answering different types of elicitation questions. We compare the prediction accuracy of our framework when adopting various information criteria that evaluate the expected information gain from a question. Our experiments show carefully designed information criteria are much more efficient, i.e., they arrive at the correct answer using fewer queries, than randomly asking questions given the budget constraint.
@cite_0: Stated preference elicitation techniques, such as discrete choice experiments and best-worst scaling, are now widely used in health research to explore the public’s choices and preferences. In this paper, we propose an alternative stated preference elicitation technique, which we refer to as ‘trio-wise’. We explain this new technique, its relative advantages, modeling framework, and how it compares to the best-worst scaling method. To better illustrate the differences and similarities, we utilize best-worst scaling Case 2, where individuals make best and worst (most and least) choices for the attribute levels that describe a single profile. We demonstrate this new preference elicitation technique using an empirical case study that explores preferences among the general public for ways to involve them in decisions concerning the health care system. Our findings show that the best-worst scaling and trio-wise preference elicitation techniques both retrieve similar preferences. However, the capability of our trio-wise method to provide additional information on the strength of rank preferences and its ability to accommodate indifferent preferences lead us to prefer it over the standard best-worst scaling technique.
@cite_9: This paper discusses General Random Utility Models (GRUMs) . These are a class of parametric models that generate partial ranks over alternatives given attributes of agents and alternatives. We propose two preference elicitation scheme for GRUMs developed from principles in Bayesian experimental design, one for social choice and the other for personalized choice. We couple this with a general Monte-Carlo-Expectation-Maximization (MC-EM) based algorithm for MAP inference under GRUMs. We also prove uni-modality of the likelihood functions for a class of GRUMs. We examine the performance of various criteria by experimental studies, which show that the proposed elicitation scheme increases the precision of estimation.
@cite_8: While voting schemes provide an effective means for aggregating preferences, methods for the effective elicitation of voter preferences have received little attention. We address this problem by first considering approximate winner determination when incomplete voter preferences are provided. Exploiting natural scoring metrics, we use max regret to measure the quality or robustness of proposed winners, and develop polynomial time algorithms for computing the alternative with minimax regret for several popular voting rules. We then show how minimax regret can be used to effectively drive incremental preference vote elicitation and devise several heuristics for this process. Despite worst-case theoretical results showing that most voting protocols require nearly complete voter preferences to determine winners, we demonstrate the practical effectiveness of regret-based elicitation for determining both approximate and exact winners on several real-world data sets.
As a key role in preference elicitation, information criteria have been widely investigated for different applications. Standard information criteria include D-optimality (used in ) and E-optimality. @cite_0 and @cite_8 use minimax-regret-based criterion for stable matching and aggregation respectively. @cite_9 proposed yet another criterion, defined on the certainty of the least certain pairwise comparison over the intrinsic utilities (part of the parameter of their general random utility models) of all alternatives. Our MPC criterion extends the criterion by @cite_9 . To predict a single agent's top @math preference, we search over a subset of all pairwise comparisons (see ). To help make a group decision, we search over all pairwise comparisons of all agents in the key group to find the least certain pairwise comparison (Equation ).