At a country fair in Plymouth in 1906, 787 visitors paid a few pence each to guess the weight of an ox after it had been slaughtered and dressed. The statistician Francis Galton collected the tickets, and in 1907 he published what he found in the journal Nature. The middle guess was 1,207 pounds. The true weight was 1,198 pounds. The crowd was off by less than one percent, which was better than almost any individual, including the cattle experts present.
Why averaging works
Every guess consists of two parts: a truth component and an error. If the errors of different people are independent, some are too high and some are too low, and they cancel when you take a mean or a median. The more independent guesses you collect, the closer the error of the group gets to zero. Mathematically, the error of the average is always smaller than the average of the individual errors, unless everyone makes exactly the same error. This is the same logic as the diversity prediction theorem, which Scott Page has described: the crowd's squared error equals the average individual squared error minus the diversity of the guesses.
James Surowiecki popularised this in his 2004 book The Wisdom of Crowds. He named four conditions: diversity of opinion, independence of judgement, decentralisation and a way to aggregate the individual guesses. Break one of them and the crowd gets worse.
When it fails
- Shared bias. If everyone is misled by the same illusion, the average inherits it. A crowd shown the same anchor will cluster around it.
- Social influence. In experiments by Jan Lorenz and colleagues in 2011, groups who saw each other's answers became more confident and less diverse, and the group's accuracy did not improve. Independence is the key.
- No skill at all. If nobody knows anything about the subject, the average of random guesses is still random.
The crowd in your head
The most surprising result is that you can be your own crowd. In 2008 Edward Vul and Harold Pashler asked people the same factual question twice, either immediately or three weeks apart, and found that the average of the two guesses was more accurate than either one alone. The gain was larger when the two guesses were separated in time, as though the second guess came from a somewhat different person. A later study by Stefan Herzog and Ralph Hertwig showed that deliberately taking a second guess from the opposite perspective ("assume your first answer was wrong; what might be another answer?") produced the same benefit.
In practice this gives you a simple method for estimates that matter. Make a guess, wait or think of a different approach, make a second guess, and take something between them. If the two guesses are far apart, that is itself information: your uncertainty is large and you should say so.
What this means for the games
In games like Gut Total and Fair Share, you are the only one in the crowd, so the classic technique is the one above: estimate in more than one way. Count a few items, scale up and then compare with your overall impression. If the two figures differ by a large factor, look at the assumption that is most uncertain.
The result also changes how you should use the numbers elsewhere. A group's median guess for the number of sweets in a jar, the time of a project or the date of an event is a good place to start. A group's confident disagreement is a warning that the thing is harder to know than it looked.