Jury theorem: Difference between revisions
Erel Segal (talk | contribs) |
Erel Segal (talk | contribs) |
||
Line 6: | Line 6: | ||
== Setting == |
== Setting == |
||
The premise of all jury theorems is that there is an [[Objective truth|''objective truth'']], which is unknown to the voters. Most theorems focus on binary |
The premise of all jury theorems is that there is an [[Objective truth|''objective truth'']], which is unknown to the voters. Most theorems focus on ''binary issues'' (issues with two possible states), for example, whether a certain [[defendant]] is guilty or innocent, whether a certain [[stock]] is going to rise or fall, etc. There are <math>n</math> voters (or jurors), and their goal is to reveal the truth. Each voter has an [[Opinion|''opinion'']] about which of the two options is correct. The opinion of each voter is either correct (i.e., equals the true state), or wrong (i.e., differs than the true state). This is in contrast to other settings of [[voting]], in which the opinion of each voter represents his/her subjective preferences and is thus always "correct" for this specific voter. The opinion of a voter can be considered a [[random variable]]: for each voter, there is a positive probability that his opinion equals the true state. |
||
The group decision is determined by the [[majority rule|''majority rule'']]. For example, if a majority of voters says "guilty" then the decision is "guilty", while if a majority says "innocent" then the decision is "innocent". To avoid |
The group decision is determined by the [[majority rule|''majority rule'']]. For example, if a majority of voters says "guilty" then the decision is "guilty", while if a majority says "innocent" then the decision is "innocent". To avoid ties, it is often assumed that the number of voters <math>n</math> is odd. Alternatively, if <math>n</math> is even, then ties are broken by tossing a [[fair coin]]. |
||
Jury theorems |
Jury theorems are interested in the ''probability of correctness'' - the probability that the majority decision coincides with the objective truth. Typical jury theorems make two kinds of claims on this probability:<ref name=":2" /> |
||
# ''Growing reliability'': the probability of correctness is larger when the group is larger. |
# ''Growing reliability'': the probability of correctness is larger when the group is larger. |
||
Line 20: | Line 20: | ||
== Unconditional independence and competence == |
== Unconditional independence and competence == |
||
{{Main|Condorcet's jury theorem}} |
{{Main|Condorcet's jury theorem}} |
||
Condorcet |
Condorcet's jury theorem makes the following three assumptions: |
||
# ''Unconditional independence'': the voters make up their minds independently. In other words, their opinions are [[Independence (probability theory)|independent random variables]]. |
# ''Unconditional independence'': the voters make up their minds independently. In other words, their opinions are [[Independence (probability theory)|independent random variables]]. |
||
Line 27: | Line 27: | ||
Under these three assumptions, he proved both growing-reliability and crowd-infallibility. |
Under these three assumptions, he proved both growing-reliability and crowd-infallibility. |
||
== Truth- |
== Truth-sensitive independence and competence == |
||
In binary decision problems, there is often one option that is easier to detect that the other one. For example, it may be easier to detect that a defendant is guilty (as there is clear evidence for guilt) than to detect that he is innocent. In this case, the probability that the opinion of a single voter is correct is represented by two different numbers: probability given that option #1 is correct, and probability given that option #2 is correct. This also implies that opinions of different voters are [[Correlation and dependence|correlated]]. This motivates the following relaxations of the above assumptions: |
In binary decision problems, there is often one option that is easier to detect that the other one. For example, it may be easier to detect that a defendant is guilty (as there is clear evidence for guilt) than to detect that he is innocent. In this case, the probability that the opinion of a single voter is correct is represented by two different numbers: probability given that option #1 is correct, and probability given that option #2 is correct. This also implies that opinions of different voters are [[Correlation and dependence|correlated]]. This motivates the following relaxations of the above assumptions: |
||
Line 41: | Line 41: | ||
* [[Deliberation]] among voters; |
* [[Deliberation]] among voters; |
||
* [[Peer pressure]]; |
* [[Peer pressure]]; |
||
* False evidence (e.g. a guilty defendant that |
* False evidence (e.g. a guilty defendant that excells at pretending to be innocent); |
||
* Other external conditions (e.g. poor weather affecting their judgement). |
* Other external conditions (e.g. poor weather affecting their judgement). |
||
Revision as of 05:54, 27 May 2021
A jury theorem is a mathematical theorem proving that, under certain assumptions, a decision attained using majority voting in a large group is more likely to be correct than a decision attained by a single expert. It serves as a formal argument for the idea of wisdom of the crowd, and for democracy in general.[1]
The first and most famous jury theorem is Condorcet's jury theorem. It assumes that all voters have independent probabilities to vote for the correct alternative, these probabilities are larger than 1/2, and are the same for all voters. Under these assumptions, the probability that the majority decision is correct is strictly larger when the group is larger; and when the group size tends to infinity, the probability that the majority decision is correct tends to 1.
There are many other jury theorems, relaxing some or all of these assumptions.
Setting
The premise of all jury theorems is that there is an objective truth, which is unknown to the voters. Most theorems focus on binary issues (issues with two possible states), for example, whether a certain defendant is guilty or innocent, whether a certain stock is going to rise or fall, etc. There are voters (or jurors), and their goal is to reveal the truth. Each voter has an opinion about which of the two options is correct. The opinion of each voter is either correct (i.e., equals the true state), or wrong (i.e., differs than the true state). This is in contrast to other settings of voting, in which the opinion of each voter represents his/her subjective preferences and is thus always "correct" for this specific voter. The opinion of a voter can be considered a random variable: for each voter, there is a positive probability that his opinion equals the true state.
The group decision is determined by the majority rule. For example, if a majority of voters says "guilty" then the decision is "guilty", while if a majority says "innocent" then the decision is "innocent". To avoid ties, it is often assumed that the number of voters is odd. Alternatively, if is even, then ties are broken by tossing a fair coin.
Jury theorems are interested in the probability of correctness - the probability that the majority decision coincides with the objective truth. Typical jury theorems make two kinds of claims on this probability:[1]
- Growing reliability: the probability of correctness is larger when the group is larger.
- Crowd infallibility: the probability of correctness goes to 1 when the group size goes to infinity.
Claim 1 is often called the non-asymptotic part and claim 2 is often called the asymptotic part of the jury theorem.
Obviously, these claims are not always true, but they are true under certain assumptions on the voters. Different jury theorems make different assumptions.
Unconditional independence and competence
Condorcet's jury theorem makes the following three assumptions:
- Unconditional independence: the voters make up their minds independently. In other words, their opinions are independent random variables.
- Unconditional competence: the probability that the opinion of a single voter coincides with the objective truth is larger than 1/2 (i.e., the voter is smarter than a random coin-toss).
- Uniformity: all voters have the same probability of being correct.
Under these three assumptions, he proved both growing-reliability and crowd-infallibility.
Truth-sensitive independence and competence
In binary decision problems, there is often one option that is easier to detect that the other one. For example, it may be easier to detect that a defendant is guilty (as there is clear evidence for guilt) than to detect that he is innocent. In this case, the probability that the opinion of a single voter is correct is represented by two different numbers: probability given that option #1 is correct, and probability given that option #2 is correct. This also implies that opinions of different voters are correlated. This motivates the following relaxations of the above assumptions:
- Conditional independence: for each of the two options, the voters' opinions given that this option is the true one are independent random variables.
- Conditional competence: for each of the two options, the probability that a single voter's opinion is correct given that this option is true is larger than 1/2.
- Conditional uniformity: for each of the two options, all voters have the same probability of being correct given that this option is true.
Growing-reliability and crowd-infallibility continue to hold under these weaker assumptions.[1]
Correlated votes
In addition to the dependence on the true option, there are many other reasons for which voters' opinions may be correlated. For example:
- Deliberation among voters;
- Peer pressure;
- False evidence (e.g. a guilty defendant that excells at pretending to be innocent);
- Other external conditions (e.g. poor weather affecting their judgement).
Negative result
In general, the growing-reliability claim fails when the votes are correlated.[2] As an example, let be the probability of a juror voting for the correct alternative and be the (second-order) correlation coefficient between any two correct votes. If all higher-order correlation coefficients in the Bahadur representation[3] of the joint probability distribution of votes equal to zero, and is an admissible pair, then the probability of the jury collectively reaching the correct decision under simple majority is given by:
where is the regularized incomplete beta function.
Example: Take a jury of three jurors , with individual competence and second-order correlation . Then . The competence of the jury is lower than the competence of a single juror, which equals to . Moreover, enlarging the jury by two jurors decreases the jury competence even further, . Note that and is an admissible pair of parameters. For and , the maximum admissible second-order correlation coefficient equals .
The above example shows that when the individual competence is low but the correlation is high:
- The collective competence under simple majority may fall below that of a single juror;
- Enlarging the jury may decrease its collective competence.
The above result is due to Kaniovski and Zaigraev, who discuss optimal jury design for homogenous juries with correlated votes.[2]
Positive results
There are jury theorems that do not require voter independence, but take into account the degree to which votes may be correlated.[4][5]
Another approach is to consider a problem-specific jury theorem.[6][7]
Non-uniform probabilities
Condorcet's theorem assumes that all voters have the same competence, i.e., the probability of deciding correctly is uniform among all voters. In practice, different voters have different competence levels.
A stronger version of the theorem requires only that the average of the individual competence levels of the voters (i.e. the average of their individual probabilities of deciding correctly) is slightly greater than half.[8]
Indirect majority systems
Condorcet's theorem considers a direct majority system, in which all votes are counted directly towards the final outcome. Many countries use an indirect majority system, in which the voters are divided into groups. The voters in each group decide on an outcome by an internal majority vote; then, the groups decide on the final outcome by a majority vote among them. For example,[9] suppose there are 15 voters. In a direct majority system, a decision is accepted whenever at least 8 votes support it. Suppose now that the voters are grouped into 3 groups of size 5 each. A decision is accepted whenever at least 2 groups support it, and in each group, a decision is accepted whenever at least 3 voters support it. Therefore, a decision may be accepted even if only 6 voters support it.
Boland, Proschan and Tong[10] prove that, when the voters are independent and p>1/2, a direct majority system - as in Condorcet's theorem - always has a higher chance of accepting the correct decision than any indirect majority system.
Berg and Paroush[11] consider multi-tier voting hierarchies, which may have several levels with different decision-making rules in each level. They study the optimal voting structure, and compares the competence against the benefit of time-saving and other expenses.
More than two options
Most theorems do not directly apply to decisions between more than two outcomes. This critical limitation was in fact recognized by Condorcet (see Condorcet's paradox), and in general it is very difficult to reconcile individual decisions between three or more outcomes (see Arrow's theorem), although List and Goodin[12] and Everaere, Konieczny and Marquis[13] present evidence to the contrary. This limitation may also be overcome by means of a sequence of votes on pairs of alternatives, as is commonly realized via the legislative amendment process. (However, as per Arrow's theorem, this creates a "path dependence" on the exact sequence of pairs of alternatives; e.g., which amendment is proposed first can make a difference in what amendment is ultimately passed, or if the law—with or without amendments—is passed at all.)
Strategic voting
The behaviour that everybody in the jury votes according to his own beliefs might not be a Nash equilibrium under certain circumstances.[14]
Limitations
The notion of "correctness" may not be meaningful when making policy decisions, as opposed to deciding questions of fact.[citation needed] Some defenders of the theorem hold that it is applicable when voting is aimed at determining which policy best promotes the public good, rather than at merely expressing individual preferences. On this reading, what the theorem says is that although each member of the electorate may only have a vague perception of which of two policies is better, majority voting has an amplifying effect. The "group competence level", as represented by the probability that the majority chooses the better alternative, increases towards 1 as the size of the electorate grows assuming that each voter is more often right than wrong.
Despite these objections, Condorcet's jury theorem provides a theoretical basis for democracy, even if somewhat idealized, as well as a basis of the decision of questions of fact by jury trial, and as such continues to be studied by political scientists.
Further reading
- Non-asymptotic Condorcet jury theorem.[15]
- Majority systems and the Condorcet jury theorem:[16] discusses non-homogeneous and correlated voters, and indirect majority systems.
- Evolution in collective decision making.[17]
- Law of large numbers
References
- ^ a b c Franz Dietrich and Kai Spiekermann (2019-07-19). Jury Theorems. Routledge. doi:10.4324/9781315717937-38/jury-theorems-franz-dietrich-kai-spiekermann. ISBN 978-1-315-71793-7.
- ^ a b Kaniovski, Serguei; Alexander, Zaigraev (2011). "Optimal Jury Design for Homogeneous Juries with Correlated Votes" (PDF). Theory and Decision. 71 (4): 439–459. CiteSeerX 10.1.1.225.5613. doi:10.1007/s11238-009-9170-2. S2CID 9189720.
- ^ Bahadur, R.R. (1961). "A representation of the joint distribution of responses to n dichotomous items". H. Solomon (Ed.), Studies in Item Analysis and Prediction: 158–168.
- ^ James Hawthorne. "Voting In Search of the Public Good: the Probabilistic Logic of Majority Judgments" (PDF). Archived from the original (PDF) on 2016-03-23. Retrieved 2009-04-20.
- ^ see for example: Krishna K. Ladha (August 1992). "The Condorcet Jury Theorem, Free Speech, and Correlated Votes". American Journal of Political Science. 36 (3): 617–634. doi:10.2307/2111584. JSTOR 2111584.
- ^ Dietrich, Franz (2008). "The Premises of Condorcet's Jury Theorem Are Not Simultaneously Justified". Episteme: A Journal of Social Epistemology. 5 (1): 56–73. doi:10.1353/epi.0.0023. ISSN 1750-0117.
- ^ Dietrich, Franz; Spiekermann, Kai (2013-03-01). "Epistemic democracy with defensible premises". Economics and Philosophy. 29 (1): 87–120. ISSN 0266-2671.
- ^ Bernard Grofman; Guillermo Owen; Scott L. Feld (1983). "Thirteen theorems in search of the truth" (PDF). Theory & Decision. 15 (3): 261–78. doi:10.1007/BF00125672. S2CID 50576036.
- ^ Boland, Philip J. (1989). "Majority Systems and the Condorcet Jury Theorem". Journal of the Royal Statistical Society, Series D (The Statistician). 38 (3): 181–189. doi:10.2307/2348873. ISSN 1467-9884. JSTOR 2348873.
- ^ Boland, Philip J.; Proschan, Frank; Tong, Y. L. (March 1989). "Modelling dependence in simple and indirect majority systems". Journal of Applied Probability. 26 (1): 81–88. doi:10.2307/3214318. ISSN 0021-9002. JSTOR 3214318.
- ^ Berg, Sven; Paroush, Jacob (1998-05-01). "Collective decision making in hierarchies". Mathematical Social Sciences. 35 (3): 233–244. doi:10.1016/S0165-4896(97)00047-4. ISSN 0165-4896.
- ^ Christian List and Robert Goodin (September 2001). "Epistemic democracy : generalizing the Condorcet Jury Theorem" (PDF). Journal of Political Philosophy. 9 (3): 277–306. CiteSeerX 10.1.1.105.9476. doi:10.1111/1467-9760.00128.
- ^ Patricia Everaere, Sébastien Konieczny and Pierre Marquis (August 2010). "The Epistemic View of Belief Merging: Can We Track the Truth?" (PDF). Proceedings of the 19th European Conference on Artificial Intelligence (ECAI'10): 621–626. CiteSeerX 10.1.1.298.3965. doi:10.3233/978-1-60750-606-5-621.
- ^ Austen-Smith, David; Banks, Jeffrey S. (1996). "Information aggregation, rationality, and the Condorcet Jury Theorem" (PDF). American Political Science Review. 90 (1): 34–45. doi:10.2307/2082796. JSTOR 2082796.
- ^ Ben-Yashar, Ruth; Paroush, Jacob (2000-03-01). "A nonasymptotic Condorcet jury theorem". Social Choice and Welfare. 17 (2): 189–199. doi:10.1007/s003550050014. ISSN 1432-217X. S2CID 32072741.
- ^ Boland, Philip J. (1989). "Majority Systems and the Condorcet Jury Theorem". Journal of the Royal Statistical Society, Series D (The Statistician). 38 (3): 181–189. doi:10.2307/2348873. ISSN 1467-9884. JSTOR 2348873.
- ^ "Evolution in collective decision making". Understanding Collective Decision Making: 167–192. 2017. doi:10.4337/9781783473151.00011. ISBN 9781783473151.