{"title": "Efficient and Thrifty Voting by Any Means Necessary", "book": "Advances in Neural Information Processing Systems", "page_first": 7180, "page_last": 7191, "abstract": "We take an unorthodox view of voting by expanding the design space to include both the elicitation rule, whereby voters map their (cardinal) preferences to votes, and the aggregation rule, which transforms the reported votes into collective decisions. Intuitively, there is a tradeoff between the communication requirements of the elicitation rule (i.e., the number of bits of information that voters need to provide about their preferences) and the efficiency of the outcome of the aggregation rule, which we measure through distortion (i.e., how well the utilitarian social welfare of the outcome approximates the maximum social welfare in the worst case). Our results chart the Pareto frontier of the communication-distortion tradeoff.", "full_text": "Ef\ufb01cient and Thrifty Voting by Any Means Necessary\n\nDebmalya Mandal\nColumbia University\n\ndm3557@columbia.edu\n\nNisarg Shah\n\nUniversity of Toronto\n\nnisarg@cs.toronto.edu\n\nAriel D. Procaccia\n\nCarnegie Mellon University\narielpro@cs.cmu.edu\n\nDavid P. Woodruff\n\nCarnegie Mellon University\ndwoodruf@cs.cmu.edu\n\nAbstract\n\nWe take an unorthodox view of voting by expanding the design space to include\nboth the elicitation rule, whereby voters map their (cardinal) preferences to votes,\nand the aggregation rule, which transforms the reported votes into collective\ndecisions. Intuitively, there is a tradeoff between the communication requirements\nof the elicitation rule (i.e., the number of bits of information that voters need to\nprovide about their preferences) and the ef\ufb01ciency of the outcome of the aggregation\nrule, which we measure through distortion (i.e., how well the utilitarian social\nwelfare of the outcome approximates the maximum social welfare in the worst case).\nOur results chart the Pareto frontier of the communication-distortion tradeoff.\n\n1\n\nIntroduction\n\nAI systems are increasingly being used to make decisions that have an impact on people and society.\nThere is much discussion of ways to ensure that such systems re\ufb02ect appropriate societal values, but\nit is often unclear what the right choices are [1]. A promising direction is to design systems that\nincorporate and aggregate people\u2019s opinions, by building on work in social choice theory [2, 3, 4].\nWhile the origins of the \ufb01eld can be traced back to the contributions of Condorcet [5] and others in the\n18th Century, it was founded in its modern form in the 20th Century. With his famous impossibility\nresult, Arrow [6] pioneered the axiomatic approach to voting, in which voting rules that aggregate\nranked preferences of individuals are compared qualitatively based on the axiomatic desiderata they\nsatisfy or violate. This approach underlies most of the work on voting in social choice theory [see,\ne.g., 7, 8].\nBy contrast, research in computational social choice [9] has put more emphasis on quantitative\nevaluation of voting rules. In particular, Procaccia and Rosenschein [10] introduced the implicit\nutilitarian voting framework, in which it is assumed that individuals (a.k.a. voters) have underlying\ncardinal utilities for the different alternatives, and express ranked preferences that are consistent with\ntheir utilities. The goal is to choose an alternative that maximizes (utilitarian) social welfare \u2014 the sum\nof utilities \u2014 by relying on the reported rankings as a proxy for the latent utilities. Speci\ufb01cally, voting\nrules are compared by their distortion, which is the worst-case ratio of the maximum social welfare to\nthe social welfare of the alternative they choose. The implicit utilitarian voting approach has received\nsigni\ufb01cant attention in the past decade [11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25], and\nvoting rules based on it have been deployed on the online voting platform robovote.org.\nBenad\u00e8 et al. [13] observe that implicit utilitarian voting has another advantage: it allows comparing\nnot only voting rules that aggregate ranked preferences, but also voting rules that aggregate other\ntypes of ballots, which they refer to as input formats. They further argue that we can associate each\ninput format with the best rule for aggregating votes in that format, and ultimately compare the\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\finput formats themselves based on the lowest distortion they make possible. They also introduce a\nnew input format, threshold approval, whereby each voter is asked to report whether her utility for\neach alternative is above or below a given threshold; this input format allows achieving logarithmic\ndistortion. The results of Benad\u00e8 et al. [13] beg the question: why should we set only a single\nthreshold? What if we set two thresholds and ask each voter to report whether her utility for each\nalternative is below the lower threshold, between the two thresholds, or above the higher threshold?\nWhat if we set \ufb01ve thresholds? Or a million for that matter? Intuitively there is a tradeoff between the\nnumber of thresholds and the distortion that can be achieved. However, perhaps adding thresholds is\nnot the most ef\ufb01cient way to drive down distortion; there may be other input formats that encapsulate\nmore useful information. (Spoiler alert: this is indeed the case.)\nOur goal in this paper is to characterize the optimal tradeoff between elicitation and distortion.\nRanking always asks voters to rank their alternatives and always asks for the same amount of\ninformation from the voters. On the other hand, consider the input format threshold approval. The\nlarger the number of thresholds, the \ufb01ner the information we elicit about voter utilities, and the\nlower the distortion. This is an example of a tradeoff between the amount of information voters are\nrequired to report, and the distortion that can be achieved. As we elicit more information from voters\nabout their utilities, we should be able to achieve lower distortion. But exactly how low? To answer\nthis question, we need a precise way to reason about the complexity of vote elicitation. We use the\nnomenclature of communication complexity [26], and, in particular, examine the number of bits\nneeded to report a vote. Note that this is simply the logarithm of the number of possible votes that a\nvoter can provide in a given input format. Hence, plurality votes that ask a voter to report which of\nthe m alternatives is her top choice contain log m bits of information, while ranked preferences that\nask a voter to rank all m alternatives contain log m! = \u0398(m log m) bits of information. Our main\nresearch question is this:\nFor any k, given a budget of at most k bits per vote, what is the minimum distortion any voting rule\ncan achieve?\n\n1.1 Our Results\n\nBefore outlining our results, we describe our framework in a bit more detail (a formal model is\npresented in Section 2). A voting rule f is composed of two parts. Its elicitation rule \u03a0f elicits\ninformation from voters about their utilities. Essentially, it chooses a (possibly randomized) mapping\nfrom utility functions to \ufb01nitely many (say k) possible responses, and each voter uses this mapping\nto cast her vote. The communication complexity of f, denoted C(f ), is then E[log k], where the\nexpectation is due to random choices made by \u03a0f . The aggregation rule \u0393f aggregates the votes cast\nby voters to choose a single alternative (possibly in a randomized way). The distortion of f, denoted\ndist(f ), is the worst-case ratio of the maximum social welfare to the (expected) social welfare of this\nchosen alternative. The distortion is typically a function of the number of alternatives m. Our goal is\nto study the tradeoff between C(f ) and dist(f ).\nFigure 1 shows our results and positions them in the context of prior work. We note that any upper\nbound with deterministic elicitation (resp. aggregation) also serves as an upper bound with randomized\nelicitation (resp. aggregation), and the converse holds for lower bounds. For deterministic elicitation,\nit is known that plurality voting rule achieves \u0398(m2) distortion with deterministic aggregation and\nlog m communication complexity, and that it is trivial to achieve \u0398(m) distortion with randomized\naggregation and zero communication complexity [20]. Our lower bounds from Section 4 establish\nthat these are the best possible asymptotic bounds with communication complexity at most log m.\nWe show that these bounds do not hold for randomized elicitation by constructing a new voting rule\nin Section 3, RANDSUBSET, which uses randomized elicitation and achieves o(m) distortion with\no(log m) communication complexity.\nWe also propose a family of voting rules, PREFTHRESHOLD, which use deterministic elicitation and\naggregation, and can achieve d distortion with O(m log(d log m)/d) communication complexity,\nshown as the solid line in Figure 1. This is an improvement over bounds achievable by existing\ndeterministic elicitation methods (even with optimal randomized aggregation): threshold approval\n\u221a\nvoting has distortion \u2126(\nm) with communication complexity m [13], and ranked voting has dis-\ntortion \u2126(\nm) with communication complexity m log m [12]. In fact, this is also an improvement\nover the randomized elicitation version of threshold approval voting, which still has distortion\n\u2126(log m/ log logm) with communication complexity m [13].\n\n\u221a\n\n2\n\n\fFigure 1: The \ufb01gure depicts lower and upper bounds on distortion which can be achieved as a\nfunction of communication complexity. We use the following abbreviation: det = deterministic,\nrand = randomized, eli = elicitation, agg = aggregation. Red dashed lines show our lower bounds\n(Theorems 4 and 6), which apply to all voting rules using deterministic or randomized elicitation.\nRed diamonds (solid line) and inverted triangle show some of the upper bounds achieved by two\nfamilies of rules we introduce \u2014 PREFTHRESHOLD (Theorem 1) and RANDSUBSET (Theorem 2) \u2014\nwhich use deterministic and randomized elicitation, respectively, and deterministic aggregation. We\nobtain tradeoffs which Pareto-dominate the best possible tradeoffs that existing elicitation methods\n(shown in blue) \u2014 such as threshold approval voting [13] and ranked voting [12] \u2014 allow even with\nrandomized aggregation.\n\nIn Section 5, we leverage tools from multi-party communication complexity to show that the bounds\nachieved by PREFTHRESHOLD are nearly optimal: any voting rule with d distortion must have\n\u2126(m/d2) communication complexity with deterministic elicitation and \u2126(m/d3) communication\ncomplexity with randomized elicitation. These are presented as dashed lines in Figure 1. Note\nthat our upper and lower bounds differ by a factor that is almost linear or almost quadratic in d,\nand sublogarithmic in m. This implies a surprising fact: when our goal is to achieve near-constant\ndistortion, randomization cannot signi\ufb01cantly help.\n\n1.2 Related Work\n\nThere are two threads of research on implicit utilitarian voting. The \ufb01rst thread does not make any\nassumptions on utilities, other than that they are normalized [10, 11, 12, 13, 19, 20, 24, 25]. The\nsecond thread assumes that utilities are induced by a metric [14, 15, 16, 17, 18, 21, 22, 23]; this\nstructure generally enables lower distortion. Our approach is consistent with the former thread.\nIn addition to the work of Benad\u00e8 et al. [13], discussed above, an especially relevant paper is that\nof Caragiannis and Procaccia [11]. Their goal is also to achieve low distortion while keeping the\ncommunication requirements low. To that end, they employ speci\ufb01c voting techniques such as\napproving a single alternative (like plurality) or approving a subset of alternatives (like approval\nvoting) \u2014 these require log m and m bits per voter, respectively \u2014 but use what they call an\nembedding to describe how voters translate their cardinal preferences into votes. However, the key\ndifference between the work of Caragiannis and Procaccia and our work is that our design space is\nmuch larger: we simultaneously optimize both the embedding and the voting technique (together,\nthese form our elicitation rule), as well as the aggregation rule.1\n\n1That said, in this work we focus only on deterministic embeddings. That is, we study elicitation rules in\nwhich voters deterministically translate their cardinal preferences into votes, and show that the foregoing result\nis impossible to achieve in this case. We discuss implications of randomized embeddings in Section 6.\n\n3\n\nO(1)log log mlog mlog m log log mm1/4(m log m)1/3m1/3mm log mm2/3 log mm3/4 log mm log log m log mmo(log m)log mm1/4m1/3mm log log mlog mmm log log mm log mCommunication ComplexityDistortionLower and Upper BoundsLower bound: det eli, rand aggLower bound: det ranking eli, rand aggLower bound: det threshold approval eli, rand aggLower bound: rand eli, rand aggLower bound: rand threshold approval eli, rand aggUpper bound (P\u0001\u0002T\u0004\u0001s\u029c\u1d0f\u029f\u1d05): det eli, det aggUpper bound (R\u1d00\u0274\u1d05S\u1d1c\u0299s\u1d07\u1d1b): rand eli, det aggLimits and Possibilities of Implicit Utilitarian Voting\fFurther a\ufb01eld, Conitzer and Sandholm [27] study how much information about the voters\u2019 ranked\npreferences has to be elicited in order to compute the outcome under a given voting rule. By contrast,\nwe are interested in designing the voting rule, and the very way in which preferences are represented,\nin order to minimize distortion. In addition, the voting rules we design ask voters to report their\napproximate utility for their top few choices or for a randomly chosen subset of alternatives. Related\nideas have been explored previously [28] or in parallel [29] in the computational social choice\nliterature, albeit in fundamentally different models.\nAnother loosely related line of work was initiated by Balcan and Harvey [30] and Badanidiyuru et al.\n[31]. Their goal is to sketch combinatorial valuation functions, that is, to encode such functions using\na polynomial number of bits in a way that the value of each subset can be recovered approximately.\nWe deal with much simpler valuation functions, but, on the other hand, are looking to achieve much\nlower communication complexity. We also note that in several query models it is standard to directly\nquery a real number [30, 31, 32, 33, 34]; by contrast, in our framework, asking for even a single real\nnumber leads to in\ufb01nite communication complexity.\n\nelicit information about voter valuations and use it to \ufb01nd an alternative with high social welfare.\n\nand let vi(S) =(cid:80)\nGiven (cid:126)v, the (utilitarian) social welfare of an alternative a is sw(a, (cid:126)v) =(cid:80)\nValuations: We adopt the standard normalization assumption that(cid:80)\n\n2 Model\nFor k \u2208 N, de\ufb01ne [k] = {1, . . . , k}. Let x \u223c D denote that random variable x has distribution D. Let\nlog denote the logarithm to base 2, and ln denote the logarithm to base e. There is a set of alternatives\nA with |A| = m, and a set of voters N = [n]. Each voter i \u2208 N is endowed with a valuation\nvi : A \u2192 R+, where vi(a) \u2265 0 represents the value of voter i for alternative a. Equivalently, we view\nvi \u2208 Rm\n+ as a vector which contains the voter\u2019s value for each alternative. We slightly abuse notation\na\u2208S vi(a) for S \u2286 A. Collectively, voter valuations are denoted (cid:126)v = (v1, . . . , vn).\ni\u2208N vi(a). Our goal is to\na\u2208A vi(a) = 1 for each i \u2208 N.\nThis can be thought of as a \u201cone voter, one vote\u201d principle for cardinal valuations as it prevents voters\nfrom overshadowing other voters by expressing very high values. 2 Let \u2206m denote the m-simplex,\n+ whose coordinates sum to 1. Hence, we have that vi \u2208 \u2206m for each\ni.e., the set of all vectors in Rm\ni \u2208 N. Given such a vector vi \u2208 \u2206m, let supp(vi) \u2286 A denote the support of vi, i.e., the set of\nalternatives a for which vi(a) > 0.\nQuery space: Consider any interaction with voter i which elicits \ufb01nitely many bits of information\nand in which the voter responds deterministically. In this interaction, the voter must provide one of\n\ufb01nitely many (say k) possible responses. We say that this interaction elicits log k bits of information.3\nIt effectively partitions \u2206m into k compartments, where the compartment corresponding to each\nresponse is the set of all valuations which would result in the voter choosing that response. In other\nwords, any interaction which elicits log k bits of information is equivalent to a query which partitions\n\u2206m into k compartments and asks the voter to pick the compartment in which her valuation belongs.\nLet Q denote the set of all queries which partition \u2206m into \ufb01nitely many compartments. For a query\nq \u2208 Q, let k(q) denote the number of compartments created by q; the number of bits elicited is\nlog k(q).4 This query space incorporates traditional elicitation methods studied in the social choice\nliterature. For instance, plurality votes (which ask voters to report their favorite alternative) use m\ncompartments, k-approval votes (which ask voters to report the set of their k favorite alternatives)\n\nuse(cid:0)m\n\n(cid:1) compartments, threshold approval votes (which ask voters to approve alternatives for which\n\ntheir value is at least a given threshold) use 2m compartments, and ranked votes (which ask voters to\nrank all alternatives) use m! compartments.\nVoting Rule: A voting rule (or simply, a rule) f consists of two parts: an elicitation rule \u03a0f and an\naggregation rule \u0393f . The (randomized) elicitation rule \u03a0f is a distribution over Q, according to which\na query q is sampled. Each voter i provides a response \u03c1i to this query, depending on her valuation\n\nk\n\n2Effectively, voters can only report the intensity of their relative preference for one alternative over another.\n3For a multi-round interaction, we can concatenate the voter\u2019s responses in different rounds. This is equivalent\n\nto a single-round interaction in which the voter is asked to provide the entire string upfront.\n\n4Note that the number of bits elicited may not be an integer, but 2 raised to the power of the number of\nbits must be an integer. We could take the ceiling to enforce an integral number of bits, and this would only\nminimally increase elicitation, but some of our lower bounds are sensitive to this non-integral formulation.\n\n4\n\n\fvi. We say that the elicitation rule is deterministic if it has singleton support (i.e., it chooses a query\ndeterministically). The (randomized) aggregation rule \u0393f takes voter responses (cid:126)\u03c1 = (\u03c11, . . . , \u03c1n) as\ninput, and returns a distribution over alternatives. We say that the aggregation rule is deterministic if\nit always returns a distribution with singleton support. Slightly abusing notation, we denote by f ((cid:126)v)\nthe (randomized) alternative returned by f when voter valuations are (cid:126)v = (v1, . . . , vn). We measure\nthe performance of f via two metrics.\n\n1. The communication complexity of f for m alternatives, denoted Cm(f ) = Eq\u223c\u03a0f log k(q), is\nthe expected number of bits of information elicited by f from each voter. We drop m from the\nsuperscript when its value is clear from the context.\n\n2. The distortion of f for m alternatives, denoted distm(f ), is the worst-case ratio of the optimal\nsocial welfare to the expected social welfare achieved by f. Again, we drop m from the\nsuperscript when its value is clear from the context. Formally,\n\ndist(f ) = sup\n\n(cid:126)v\u2208(\u2206m)n\n\nmaxa\u2208A sw(a, (cid:126)v)\n\nE(cid:98)a\u223cf ((cid:126)v) sw((cid:98)a, (cid:126)v)\n\n.\n\nWhile it is desirable for a voting rule to have low communication complexity and low distortion,\ntypically eliciting more information from voters enables achieving low distortion. Our goal is to\nunderstand the Pareto frontier of the tradeoff between communication complexity and distortion.\n\n3 Upper Bounds\n\nIn this section, we derive upper bounds on the best distortion a voting rule can achieve given an\nupper bound on its communication complexity (equivalently, this gives an upper bound on the\ncommunication complexity required to achieve a given level of distortion). We construct two\nfamilies of voting rules: PREFTHRESHOLD, which use deterministic elicitation and aggregation, and\nRANDSUBSET, which convert a given voting rule into one which uses randomized elicitation.\n\n3.1 Deterministic Elicitation, Deterministic Aggregation\n\nWe begin by designing voting rules which use deterministic elicitation and deterministic aggregation \u2014\nthe most practical combination. Caragiannis et al. [20] show that plurality achieves \u0398(m2) distortion\nwith log m communication complexity, and even voting rules that elicit ranked preferences, and thus\nhave \u0398(m log m) communication complexity, cannot achieve asymptotically better distortion.\nWe propose a novel voting rule PREFTHRESHOLDt,(cid:96), parametrized by t \u2208 [m] and (cid:96) \u2208 N. It is\npresented as Algorithm 1. Its elicitation rule asks each voter to report the set of her t most preferred\nalternatives, and for each alternative in this set, report her approximate value for it by picking one of\n(cid:96) + 1 subintervals of [0, 1]. Note that for t = 1, we use (cid:96) subintervals of [1/m, 1]; this is valid because\na voter\u2019s value for her most favorite alternative must be at least 1/m. The aggregation rule is also\nintuitive: it uses the approximate values to compute an estimated social welfare of each alternative,\nand picks an alternative with the highest estimated social welfare. The next theorem provides bounds\non the communication and distortion of PREFTHRESHOLDt,(cid:96).5\nTheorem 1. For t \u2208 [m] \\ {1} and (cid:96) \u2208 N, we have\nC(PREFTHRESHOLDt,(cid:96)) = \u0398\nFor t = 1 and (cid:96) \u2208 N, we have\n\n, dist(PREFTHRESHOLDt,(cid:96)) = O\n\nm1+2/(cid:96)/t\n\nm((cid:96) + 1)\n\n(cid:18)\n\n(cid:17)\n\n.\n\n(cid:19)\n\nt log\n\nt\n\n(cid:16)\nm1+1/(cid:96)(cid:17)\n\n(cid:16)\n\n.\n\nC(PREFTHRESHOLD1,(cid:96)) = log(m(cid:96)), dist(PREFTHRESHOLDt,(cid:96)) = O\n\nPREFTHRESHOLDt,(cid:96) offers a tradeoff between two parameters, t and (cid:96). Increasing either parameter\nincreases the communication complexity but reduces the distortion. It is easy to see that there is no\n(asymptotic) bene\ufb01t of choosing (cid:96) > log m. We make several observations.\n\n\u2022 t = 1, (cid:96) = 2 gives us subquadratic distortion of O(m\n\nthan plurality (i.e. log m + 1 bits).\n\nm) with just one more bit of elicitation\n\n\u221a\n\n5All omitted proofs are included in the supplementary material.\n\n5\n\n\fALGORITHM 1: PREFTHRESHOLDt,(cid:96), where t \u2208 [m] and (cid:96) \u2208 N.\nElicitation Rule:\n\u2022 If t > 1, create (cid:96) + 1 buckets: B0 = [0, 1/m2] and Bp = (1/m2\u22122(p\u22121)/(cid:96), 1/m2\u22122p/(cid:96)] for p \u2208 [(cid:96)].\n\u2022 If t = 1, create (cid:96) buckets: B1 = [m\u22121, m\u22121+1/(cid:96)] and Bp = (m\u22121+(p\u22121)/(cid:96), m\u22121+p/(cid:96)] for p \u2208 [(cid:96)] \\ {1}.\n\u2022 The query asks each voter i to identify set St\n(breaking ties arbitrarily), and for each a \u2208 St\n\ni of the t alternatives for which she has the highest value\ni , identify bucket index pi,a such that vi(a) \u2208 Bpi,a.\n\u2022 For each voter i \u2208 N and alternative a \u2208 A, de\ufb01ne (cid:98)vi(a) = Upi,a if a \u2208 St\ni and (cid:98)vi(a) = 0 o.w.\n\u2022 For an alternative a \u2208 A, de\ufb01ne the estimated social welfare as(cid:99)sw(a) =(cid:80)\ni\u2208N (cid:98)vi(a).\n\u2022 Return an alternative with the highest estimated social welfare, i.e.,(cid:98)a \u2208 arg maxa\u2208A(cid:99)sw(a).\n\n\u2022 For each p, let Up denote the upper endpoint of bucket Bp.\n\nAggregation Rule:\n\n\u2022 t = m1\u2212\u03b3, (cid:96) = log m gives us sublinear distortion of O(m\u03b3) (for \u03b3 \u2208 (0, 1)) with polynomial\ncommunication complexity of O(m1\u2212\u03b3 log m).\n\nlog m, (cid:96) = log m has distortion O(cid:0)\u221a\n\nlog m(cid:1) with communication o(m), and\n\n\u2022 t = m/\n\nPareto-dominates threshold approval voting, which has higher communication complexity of m\nand higher distortion of \u2126(log m/ log log m), even with randomized aggregation [13].\n\n\u221a\n\n\u2022 t = m, (cid:96) = log m leads to constant distortion with communication O(m log log m). By con-\ntrast, eliciting ranking leads to higher communication complexity of \u0398(m log m), and also\nsigni\ufb01cantly higher distortion of \u2126(\n\nm), even with randomized aggregation [12].\n\n\u221a\n\n3.2 Randomized Elicitation, Randomized Aggregation\n\nWe now present a generic approach to designing voting rules with randomized elicitation. Given\na voting rule f and an integer s \u2264 m, instead of using f to select one alternative from A directly,\nwe sample S \u2286 A with |S| = s at random and use f to select one alternative from S. Recall\nthat for p \u2208 N, Cp(f ) and distp(f ) denote the communication complexity and distortion of f for p\nalternatives, respectively.\nClearly, this approach reduces the communication complexity from Cm(f ) to Cs(f ). Its effect on\ndistortion, however, is more subtle. On the one hand, selecting an alternative from S instead of A\nresults in an inevitable loss of welfare because we can only hope to do as well as the best alternative\nin S. On the other hand, the welfare we achieve is related to the welfare of the best alternative in\nS via the factor dists(f ), which can be signi\ufb01cantly lower than distm(f ). We show that in some\ncases, this approach reduces distortion in addition to reducing communication complexity. The key\nchallenge in making this approach work is that we cannot apply f directly to select one alternative\nfrom S, as the total value of the alternatives in S need not be 1. We circumvent this obstacle by\neliciting an approximate value of vi(S) from each voter i, making a number of copies of voter i that\nis approximately proportional to vi(S) with each copy now having a total value of 1 for alternatives\nin S, and running f on the resulting instance.\n\nALGORITHM 2: RANDSUBSET(f, s), where f is a voting rule and s \u2208 [m]\nElicitation Rule:\n\u2022 Pick S \u2286 A with |S| = s uniformly at random from among all subsets of A of size s.\n4m , 2j\n\n\u2022 Partition [0, 1] into (cid:100)log(4m)(cid:101) buckets as follows: B0 =(cid:2)0, 1\n1. The bucket index pi such that vi(S) =(cid:80)\na\u2208S vi(a) \u2208 Bpi;\nvaluation (cid:98)vi de\ufb01ned as (cid:98)vi(a) = vi(a)/vi(S) for each a \u2208 S.\n\n\u2022 Ask two reports from each voter i:\n\n(cid:3), Bj =\n\n(cid:16) 2j\u22121\n\n4m\n\n(cid:105)\n\nfor j \u2208 (cid:100)log(4m)(cid:101).\n\n4m\n\n2. A response \u03c1i to the elicitation rule of f for the set of alternatives S according to the renormalized\n\nAggregation Rule:\n\u2022 Let Lp denote the lower endpoint of bucket Bp for p \u2208 (cid:100)log(4m)(cid:101) \u222a {0}.\n\u2022 Run the aggregation rule of f on an input which consists of 4m \u00b7 Lpi copies of \u03c1i for each i \u2208 N.\n\n6\n\n\fTheorem 2. For every voting rule f and s \u2208 [m], we have Cm(RANDSUBSET(f, s)) = Cs(f ) +\nlog(cid:100)log(4m)(cid:101) and distm(RANDSUBSET(f, s)) \u2264 4m\nUsing f = PREFTHRESHOLDt,(cid:96) and Theorem 1, we obtain that for s \u2208 [m], t \u2208 [s], and (cid:96) \u2208 N,\nthere is a new voting rule g = RANDSUBSET (PREFTHRESHOLDt,(cid:96), s) with\n\ns \u00b7 dists(f ).\n\n(cid:16)\n\n(cid:17)\n\nCm(g) = O (t log(s((cid:96) + 1)/t) + log log m) and distm(g) = O\n\nm \u00b7 s2/(cid:96)/t\n\n.\n\nSetting (cid:96) = log s, we get O(m/t) distortion. Then, we set s = t to minimize communication\ncomplexity to O(t log log t + log log m). This is slightly better than using PREFTHRESHOLDt,log m,\nwhich achieves O(m/t) distortion with O(t log m log m\n) communication complexity. In particular,\nfor t = O(1) this reduces communication complexity by a factor of log m/ log log m.\nAn interesting choice is t = log m\nlog log m, which leads to distortion O (m log log m/ log m) = o(m) and\ncommunication complexity O (t log log t + log log m) = o(log m). Note that this rule has random-\nized elicitation but deterministic aggregation. By contrast, we later show that with deterministic\nelicitation, no voting rule can achieve o(m) distortion with communication complexity at most log m,\neven when randomized aggregation is allowed (Theorem 4).\n\nt\n\n4 Direct Lower Bounds For Deterministic Elicitation\n\nWe now turn to deriving lower bounds on the distortion of a voting rule given an upper bound on its\ncommunication complexity (equivalently, this gives a lower bound on the communication complexity\nrequired to achieve a given level of distortion). In this section, we focus on deterministic elicitation.\nConsider a voting rule f which uses deterministic elicitation and has communication complexity at\nmost log k. Hence, the (deterministic) query of f must partition \u2206m into at most k compartments.\nWthout loss of generality, we can assume that f uses exactly k compartments. This is because if f\nuses k(cid:48) compartments where k(cid:48) < k, then we can partition some of its compartments into smaller\ncompartments and derive a new voting rule g which uses exactly k compartments, receives at least\nthe information that f receives from the voters, and simulates the aggregation rule of f to achieve\nthe same distortion. Now, establishing a lower bound on the distortion of f requires analyzing the\nfollowing game between two players, the voting rule f and the adversary.\n\n1. The voting rule f decides the partition of \u2206m into k compartments.\n2. The adversary decides the response of each voter.\n3. The voting rule f picks a winning alternative (or a distribution over winning alternatives, if\n\nits aggregation rule is randomized).\n\n4. The adversary picks valuations of voters consistent with their responses in the second step.\n\nWe use this framework to derive lower bounds on the distortion of voting rules that use deterministic\nelicitation. We \ufb01rst focus on deterministic aggregation. Perhaps the simplest such voting rule is\nplurality, which has log m communication complexity and achieves \u0398(m2) distortion. This raises\nan important question: What distortion can we achieve with deterministic elicitation, deterministic\naggregation, and communication complexity less than log m? The next lemma shows that the answer\nturns out to be disappointing.\nTheorem 3. Every voting rule that has deterministic elicitation, deterministic aggregation, and\ncommunication complexity strictly less than log m has unbounded distortion.\n\nNow, plurality has communication complexity log m and achieves \u0398(m2) distortion. Can a dif-\nferent voting rule achieve better distortion using only log m communication complexity? Perhaps\nunsurprisingly, we answer this in the negative. But the proof of this intuitive result is quite intricate.\nFurther, using randomized aggregation we can trivially achieve O(m) distortion with zero communi-\ncation complexity (by returning the uniform distribution over alternatives). One may wonder: How\nmuch information do we need from the voters to achieve sublinear distortion? It is easy to show that\neliciting plurality votes is not suf\ufb01cient. Surprisingly, we show that this holds for every log m-bit\nelicitation. That is, even with randomized aggregation, eliciting log m bits per voter is asymptotically\nno better than blindly selecting an alternative uniformly at random!\n\n7\n\n\fTheorem 4. Let f be a voting rule with deterministic elicitation and C(f ) \u2264 log m. If f uses\ndeterministic (resp. randomized) aggregation, then dist(f ) = \u2126(m2) (resp. \u2126(m)).\n\nFor deterministic aggregation, Theorem 4 shows that eliciting log m bits per voter is not suf\ufb01cient to\nachieve o(m2) distortion. By contrast, we know from Theorem 1 that we can achieve O(m) distortion\nby eliciting O(log m) bits per voter. Similarly, for randomized aggregation, Theorem 4 shows that\neliciting log m bits per voter is not suf\ufb01cient to achieve o(m) distortion. However, we can achieve\no(m) distortion if we are willing to elicit \u03c9(log m) bits per voter (Theorem 1),6 or if we are willing\nto use randomized elicitation (Theorem 2).\n\n5 Lower Bounds Through Multi-Party Communication Complexity\n\nIn this section, we leverage tools from the literature on multi-party communication complexity to\nderive lower bounds for both deterministic and randomized elicitation. Speci\ufb01cally, we derive lower\nbounds on the communication complexity of voting rules that achieve a given level of distortion. We\nbegin by reviewing existing results on multi-party communication complexity, and then derive new\nresults, which help us prove the desired lower bounds in our voting context.\n\n5.1 Setup\n\nIn multi-party communication complexity, there are t computationally omnipotent players. Each\nplayer i holds a private input Xi \u2208 Xi. The input pro\ufb01le is the vector (X1, . . . , Xt). The goal is to\ncompute the output of a function f : X1 \u00d7 X2 \u00d7 . . . \u00d7 Xt \u2192 {0, 1} on the input pro\ufb01le.\nA shared protocol \u03a0 speci\ufb01es how the players exchange information among themselves and with the\ncenter. We use the blackboard model, in which messages written by one player are visible to all other\nplayers. Let \u03a0(X1, . . . , Xt) be the random variable denoting the message transcript generated when\nall players follow the protocol on input pro\ufb01le (X1, . . . , Xt); here, the randomness is due to coin\ntosses by the players or the protocol. The communication cost of \u03a0, denoted |\u03a0|, is the maximum\nlength of \u03a0(X1, . . . , Xt) over all input pro\ufb01les (X1, . . . , Xt) and all coin tosses. Given \u03b4 \u2265 0, we\nsay that \u03a0 is a \u03b4-error protocol for f if there exists a function \u03a0out such that for every input pro\ufb01le\n(X1, . . . , Xt), Pr [\u03a0out(\u03a0(X1, . . . , Xt)) = f (X1, . . . , Xt)] \u2265 1 \u2212 \u03b4. The \u03b4-error communication\ncomplexity of f, denoted R\u03b4(f ), is the communication cost of the best \u03b4-error protocol for f.\n\n5.2 Multi-Party Fixed-Size Set-Disjointness\n\nThe main ingredient of our proof is a standard problem in multi-party communication complexity\ncalled the multi-party set-disjointness problem, denoted DISJm,t. Here, each player i holds an\narbitrary set Si from a universe of size m. The goal is to distinguish between two types of inputs.\n\n\u2022 NO inputs: The sets are pairwise disjoint, i.e., Si \u2229 Sj = \u2205 for all i (cid:54)= j.\n\u2022 YES inputs: The sets have a unique element in common, but are otherwise pairwise disjoint,\ni.e., there exists x such that Si \u2229 Sj = {x} for all i (cid:54)= j.\n\nIt is promised that the input will be one of these two types (in other words, the protocol is free\nto choose any output on an input that does not satisfy this promise). Following a series of results\n[35, 36, 37], Gronemeier [38] and Jayram [39] \ufb01nally established the optimal lower bound of \u2126(m/t).\nWe introduce a variant of this problem, which we call multi-party \ufb01xed-size set-disjointness and\ndenote FDISJm,s,t. It is almost identical to DISJm,t, except that we know each player i holds a set\nSi of a given size s. Our goal is to still determine whether the sets are pairwise disjoint (Si \u2229 Sj = \u2205\nfor all i (cid:54)= j) or pairwise uniquely intersecting (there exists x such that Si \u2229 Sj = {x} for all i (cid:54)= j).\nWe use the lower bound on R\u03b4(DISJm,t) to derive the following lower bound on R\u03b4(FDISJm,s,t).\nTheorem 5. For a suf\ufb01ciently small constant \u03b4 > 0 and m \u2265 (3/2)st, R\u03b4(FDISJm,s,t) = \u2126(s).\n\n6For t = \u03c9(1), PREFTHRESHOLDt,log m has distortion O(m/t) = o(m) and communication O(t log m).\n\n8\n\n\fTable 1: Comparison between our lower bounds (Theorem 6) and upper bounds (Theorem 1)\n\nDistortion\n\nO(m\u03b3)\n\nO(log m)\n\nO(1)\n\nLower Bounds\n\nDeterministic Elicitation Randomized Elicitation\n\n\u2126(m1\u22122\u03b3)\n\n\u2126(cid:0)m/ log2 m(cid:1)\n\n\u2126(m1\u22123\u03b3)\n\n\u2126(cid:0)m/ log3 m(cid:1)\n\nUpper Bound\n\nO(m1\u2212\u03b3 log m)\n\nO(m log log m/ log m)\n\n\u2126(m)\n\n\u2126(m)\n\nO(m log log m)\n\n5.3 Lower Bounds on the Communication Complexity of Voting Rules\n\nWe now use our lower bound on the \u03b4-error communication complexity of FDISJm,s,t to derive a\nlower bound on the communication complexity of a voting rule in terms of its distortion. We derive\ndifferent bounds depending on whether the elicitation rule of f is deterministic or randomized. For\nrandomized elicitation, our bound is weaker.\nThe key insight in the proof is that we can use a voting rule f with dist(f ) \u2264 t/2 to construct a\n\u03b4-error protocol for solving FDISJm,s,t, and hence we can use the lower bound on R\u03b4(FDISJm,s,t)\nfrom Theorem 5 to derive a lower bound on C(f ). At a high level, consider an instance (S1, . . . , St)\nof FDISJm,s,t. We ask each player i to respond to the query of f according to an arti\ufb01cial valuation\nfunction constructed using Si. We then use these responses to create an input for the aggregation rule\nof f. We show that by asking each player an additional question about the alternative returned by the\naggregation rule, and possibly running this process a number of times, we can solve FDISJm,s,t.\nTheorem 6. Consider a voting rule f with elicitation rule \u03a0f and dist(f ) = d. If \u03a0f is deterministic,\n\nthen C(f ) \u2265 \u2126(cid:0)m/d2(cid:1), and if \u03a0f is randomized, then C(f ) \u2265 \u2126(cid:0)m/d3(cid:1).\n\nFinally, Table 1 summarizes our upper and lower bounds for some special cases. Achieving sublinear\ndistortion makes polynomial communication complexity both necessary (even with randomized\naggregation) and suf\ufb01cient (even with deterministic aggregation). If dist(f ) = O(log m), our upper\nand lower bounds differ by only polylogarithmic factors. And for constant distortion, they differ by\nonly a sublogarithmic factor.\n\n6 Discussion\n\nWe initiated a formal study of the communication-distortion tradeoff in voting, but our work leaves\nmany open questions. The most immediate direction is to close the gap between our upper and\nlower bounds. Regarding our upper bounds, both families of voting rules that we introduce \u2014\nPREFTHRESHOLD and RANDSUBSET \u2014 use deterministic aggregation, and we do not have better\nupper bounds using randomized aggregation. Our lower bounds from Theorem 6 are also identical\nfor deterministic and randomized aggregation. This raises an elegant question: Can randomized\naggregation help? Also, using randomized elicitation in RANDSUBSET, we can achieve sublinear\ndistortion with communication complexity at most log m; Theorem 4 shows that this is not possible\nwith deterministic elicitation. This raises another elegant question: What is the best possible distortion\nwith randomized elicitation and communication complexity at most log m? It would also be interesting\nto improve upon our lower bounds in Section 5, potentially by using a different problem from the\nmulti-party communication complexity literature.\nTaking a broader viewpoint, we can consider more general forms of elicitation, like non-uniform\nquestions across voters, and questions adaptive to past responses. One can also explore the effect\nof imposing other restrictions on the voting rule such as truthfulness [25, 40]. On a conceptual\nlevel, perhaps the main take-away message of our paper is that it pays off to elicit and aggregate\npreferences \u201cby any means necessary,\u201d that is, potentially through highly nonstandard aggregation\nand, especially, elicitation rules. In the setting of Caragiannis and Procaccia [11], voters are software\nagents, and this is natural. But when voters are people, it is crucial to understand the implications of\nsuch unconventional approaches, both in terms of how communication complexity corresponds to\ncognitive burden, and in terms of the interpretability and transparency of aggregation rules.\n\n9\n\n\fAcknowledgments\n\nMandal was partially supported by the Post-Doctoral fellowship from the Columbia Data Science\nInstitute, and part of this work was done while he was a graduate student at Harvard University.\nProcaccia was partially supported by the National Science Foundation under grants IIS-1350598, IIS-\n1714140, CCF-1525932, and CCF-1733556; by the Of\ufb01ce of Naval Research under grants N00014-\n16-1-3075 and N00014-17-1-2428; by a J.P. Morgan AI Research Award; and by a Guggenheim\nFellowship. Shah was partially supported by the Natural Sciences and Engineering Research Council\nunder a Discovery grant. Woodruff was partially supported by the National Science Foundation under\ngrant CCF-1815840, and part of this work was done while he was visiting the Simons Institute for\nthe Theory of Computing.\n\nReferences\n[1] A. Awad, S. Dsouza, R. Kim, J. Schulz, J. Henrich, A. Shariff, J.-F. Bonnefon, and I. Rahwan.\n\nThe Moral Machine experiment. Nature, 563:59\u201364, 2018.\n\n[2] J. Greene, F. Rossi, J. Tasioulas, K. B. Venable, and B. Williams. Embedding ethical principles\nin collective decision support systems. In Proceedings of the 30th AAAI Conference on Arti\ufb01cial\nIntelligence (AAAI), pages 4147\u20134151, 2016.\n\n[3] V. Conitzer, W. Sinnott-Armstrong, J. Schaich Borg, Y. Deng, and M. Kramer. Moral decision\nmaking frameworks for arti\ufb01cial intelligence. In Proceedings of the 31st AAAI Conference on\nArti\ufb01cial Intelligence (AAAI), pages 4831\u20134835, 2017.\n\n[4] R. Noothigattu, S. S. Gaikwad, E. Awad, S. Dsouza, I. Rahwan, P. Ravikumar, and A. D.\nProcaccia. A voting-based system for ethical decision making. In Proceedings of the 32nd\nAAAI Conference on Arti\ufb01cial Intelligence (AAAI), pages 1587\u20131594, 2018.\n\n[5] Marquis de Condorcet. Essai sur l\u2019application de l\u2019analyse \u00e0 la probabilit\u00e9 de d\u00e9cisions rendues\nImprimerie Royal, 1785. Facsimile published in 1972 by Chelsea\n\n\u00e0 la pluralit\u00e9 de voix.\nPublishing Company, New York.\n\n[6] K. Arrow. Social Choice and Individual Values. Wiley, 1951.\n\n[7] K. J. Arrow, A. Sen, and K. Suzumura. Handbook of Social Choice and Welfare, volume 2.\n\nElsevier, 2010.\n\n[8] A. Sen. Social choice theory. Handbook of Mathematical Economics, 3:1073\u20131181, 1986.\n\n[9] F. Brandt, V. Conitzer, U. Endress, J. Lang, and A. D. Procaccia, editors. Handbook of\n\nComputational Social Choice. Cambridge University Press, 2016.\n\n[10] A. D. Procaccia and J. S. Rosenschein. The distortion of cardinal preferences in voting. In\nProceedings of the 10th International Workshop on Cooperative Information Agents (CIA),\npages 317\u2013331, 2006.\n\n[11] I. Caragiannis and A. D. Procaccia. Voting almost maximizes social welfare despite limited\n\ncommunication. Arti\ufb01cial Intelligence, 175(9\u201310):1655\u20131671, 2011.\n\n[12] C. Boutilier, I. Caragiannis, S. Haber, T. Lu, A. D. Procaccia, and O. Sheffet. Optimal social\n\nchoice functions: A utilitarian view. Arti\ufb01cial Intelligence, 227:190\u2013213, 2015.\n\n[13] G. Benad\u00e8, S. Nath, A. D. Procaccia, and N. Shah. Preference elicitation for participatory\nbudgeting. In Proceedings of the 31st AAAI Conference on Arti\ufb01cial Intelligence (AAAI), pages\n376\u2013382, 2017.\n\n[14] E. Anshelevich, O. Bhardwaj, E. Elkind, J. Postl, and P. Skowron. Approximating optimal\n\nsocial choice under metric preferences. Arti\ufb01cial Intelligence, 264:27\u201351, 2018.\n\n[15] E. Anshelevich and J. Postl. Randomized social choice functions under metric preferences.\n\nJournal of Arti\ufb01cial Intelligence Research, 58:797\u2013827, 2017.\n\n10\n\n\f[16] S. Gross, E. Anshelevich, and L. Xia. Vote until two of you agree: Mechanisms with small\ndistortion and sample complexity. In Proceedings of the 31st AAAI Conference on Arti\ufb01cial\nIntelligence (AAAI), pages 544\u2013550, 2017.\n\n[17] A. Goel, A. K. Krishnaswamy, and K. Munagala. Metric distortion of social choice rules: Lower\nbounds and fairness properties. In Proceedings of the 18th ACM Conference on Economics and\nComputation (EC), pages 287\u2013304, 2017.\n\n[18] Y. Cheng, S. Dughmi, and D. Kempe. On the distortion of voting with multiple representative\ncandidates. In Proceedings of the 32nd AAAI Conference on Arti\ufb01cial Intelligence (AAAI),\npages 973\u2013980, 2018.\n\n[19] G. Benad\u00e9, A. D. Procaccia, and M. Qiao. Low-distortion social welfare functions. In Proceed-\n\nings of the 33rd AAAI Conference on Arti\ufb01cial Intelligence (AAAI), 2019. Forthcoming.\n\n[20] I. Caragiannis, S. Nath, A. D. Procaccia, and N. Shah. Subset selection via implicit utilitarian\n\nvoting. Journal of Arti\ufb01cial Intelligence Research, 58:123\u2013152, 2017.\n\n[21] A. Borodin, O. Lev, N. Shah, and T. Strangway. Primarily about primaries. In Proceedings of\n\nthe 33rd AAAI Conference on Arti\ufb01cial Intelligence (AAAI), 2019. Forthcoming.\n\n[22] M. Ghodsi, M. Lati\ufb01an, and M. Seddighin. On the distortion value of the elections with\nabstention. In Proceedings of the 33rd AAAI Conference on Arti\ufb01cial Intelligence (AAAI), 2019.\nForthcoming.\n\n[23] M. Feldman, A. Fiat, and I. Golumb. On voting and facility location. In Proceedings of the\n\n17th ACM Conference on Economics and Computation (EC), pages 269\u2013286, 2016.\n\n[24] U. Bhaskar and A. Ghosh. On the welfare of cardinal voting mechanisms. In Proceedings of\nthe 38th IARCS Annual Conference on Foundations of Software Technology and Theoretical\nComputer Science (FSTTCS), pages 27:1\u201327:22, 2018.\n\n[25] U. Bhaskar, V. Dani, and A. Ghosh. Truthful and near-optimal mechanisms for welfare\nmaximization in multi-winner elections. In Proceedings of the 32nd AAAI Conference on\nArti\ufb01cial Intelligence (AAAI), pages 925\u2013932, 2018.\n\n[26] E. Kushilevitz and N. Nisan. Communication Complexity. Cambridge University Press, 1996.\n\n[27] V. Conitzer and T. Sandholm. Communication complexity of common voting rules. In Pro-\nceedings of the 6th ACM Conference on Economics and Computation (EC), pages 78\u201387,\n2005.\n\n[28] Y. Filmus and J. Oren. Ef\ufb01cient voting via the top-k elicitation scheme: A probabilistic\napproach. In Proceedings of the 15th ACM Conference on Economics and Computation (EC),\npages 295\u2013312, 2014.\n\n[29] M. Bentert and P. Skowron. Comparing election methods where each voter ranks only few\n\ncandidates. arXiv:1901.10848, 2019.\n\n[30] M.-F. Balcan and N. J. Harvey. Submodular functions: Learnability, structure, and optimization.\n\nSIAM Journal on Computing, 47(3):703\u2013754, 2018.\n\n[31] A. Badanidiyuru, S. Dobzinski, H. Fu, R. Kleinberg, N. Nisan, and T. Roughgarden. Sketching\nvaluation functions. In Proceedings of the 23rd Annual ACM-SIAM Symposium on Discrete\nAlgorithms (SODA), pages 1025\u20131035, 2012.\n\n[32] E. Balkanski, A. Rubinstein, and Y. Singer. The limitations of optimization from samples.\nIn Proceedings of the 49th Annual ACM Symposium on Theory of Computing (STOC), pages\n1016\u20131027, 2017.\n\n[33] U. Feige, V. S. Mirrokni, and J. Vondr\u00e1k. Maximizing non-monotone submodular functions.\n\nSIAM Journal on Computing, 40(4):1133\u20131153, 2011.\n\n11\n\n\f[34] V. S. Mirrokni, M. Schapira, and J. Vondr\u00e1k. Tight information-theoretic lower bounds for\nwelfare maximization in combinatorial auctions. In Proceedings of the 9th ACM Conference on\nElectronic Commerce (EC), pages 70\u201377, 2008.\n\n[35] Z. Bar-Yossef, T. S. Jayram, R. Kumar, and D. Sivakumar. An information statistics approach\nto data stream and communication complexity. Journal of Computer and System Sciences, 68\n(4):702\u2013732, 2004.\n\n[36] N. Alon, Y. Matias, and M. Szegedy. The space complexity of approximating the frequency\n\nmoments. Journal of Computer and System Sciences, 58(1):137\u2013147, 1999.\n\n[37] A. Chakrabarti, S. Khot, and X. Sun. Near-optimal lower bounds on the multi-party communi-\ncation complexity of set disjointness. In Proceedings of the 18th IEEE Annual Conference on\nComputational Complexity (CCC), pages 107\u2013117, 2003.\n\n[38] A. Gronemeier. Asymptotically optimal lower bounds on the NIH-multi-party information\ncomplexity of the AND-function and disjointness. In Proceedings of the 26th International\nSymposium on Theoretical Aspects of Computer Science (STACS), pages 505\u2013516, 2009.\n\n[39] T. S. Jayram. Hellinger strikes back: A note on the multi-party information complexity of\nAND. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and\nTechniques, pages 562\u2013573. 2009.\n\n[40] A. Gibbard. Manipulation of schemes that mix voting with chance. Econometrica, 45:665\u2013681,\n\n1977.\n\n12\n\n\f", "award": [], "sourceid": 3904, "authors": [{"given_name": "Debmalya", "family_name": "Mandal", "institution": "Columbia University"}, {"given_name": "Ariel", "family_name": "Procaccia", "institution": "Carnegie Mellon University"}, {"given_name": "Nisarg", "family_name": "Shah", "institution": "University of Toronto"}, {"given_name": "David", "family_name": "Woodruff", "institution": "Carnegie Mellon University"}]}