An Introduction to the Wisdom of the Crowd
The Concept of the Wisdom of the Crowd
James Surowiecki in his book "The Wisdom of Crowds" relates the story of how the concept was first discovered.
Dalton made key discoveries in biology such as color blindness. However, he was surprised in his later years to find that the average, aggregate guess of a large crowd about the weight of a steer. While the individual guesses were often far off, the average of all the groups’ guesses was only one pound off from the actual weight of a more than 1,200 pound animal, a rounding error by most accounts. This led to the idea that the crowd in aggregate can be more right any individual in it, hence “the wisdom of the crowd”.
His book "The Wisdom of Crowds" gives a number of other examples where the aggregate guess or solution was more correct or closer to optimal than even an expert's opinion.
When Is the Crowd Most Likely to Be Right?
James Surowiecki says in his book the crowd is most likely to be correct when the crowd is diverse. This isn’t demographic diversity per se. After all, a room full of Ivy League lawyers of all shapes and sizes is going to have a similar opinion on many matters, while a mix of white men of different classes, professions and ages will have a broader variety of opinions and thus more accurate consensus view. The crowd of political activists trying to put together an estimate or consensus will be biased in one direction in comparison to the opinions of people of many different persuasions. If everyone in the group is like minded, the crowd itself is likely to be wrong.
The second requirement for the crowd to display wisdom (in aggregate) is independence. Each person must be able to give their opinion without others criticizing them for it (so they change their vote) or electioneering for an outcome. You can have some collaboration to share ideas, but too much collaboration will skew the results. Another variation of this is the extra weight given to the supposed experts who then shape the opinion of others or dictate the solution that the majority votes for – though the average aggregate answer is likely to be more accurate than the expert’s opinion by itself. Thus the promotion of the expert to a super-voter, outweighing and shaping the others, increases the odds the group is wrong.
You can maximize both diversity and independence by using large groups. The larger the group, on average, the more diversity in opinions you’ll see. And a larger group is harder for one person to try to influence directly or indirectly. The limitation on group size is manageability. If it is too large to manage, such as being unable to get everyone’s input in a timely manner, use a smaller group of similar demographic diversity.
What Types of Tasks Can Crowds Handle?
The wisdom of the crowd works best when applied to three types of tasks: cognition, coordination and cooperation.
Cognition tasks rely on collection of values that are averaged to get a result. Dalton’s weight guesstimates for the steer is the classic example. Similar tasks include guessing how many candies are in a container, the location of a sunken ship, the number of items produced per year and so forth.
Coordination tasks are those where the group figures out how to work together. Children figuring out on their own how to play a newly invented game is a classic example. Stock markets and betting rings are another. This means the stock market, as long as there is no insider trading or false information, is an excellent example of wisdom of the crowds. The tendency of people betting on who will win the presidency often being more accurate as a group in determining the winner over exit polls is another. People working together in a company and sharing an ice rink is more types of coordination tasks.
Cooperation is the type of task where you ask people to work together for a common benefit but contrary to their self interest. Asking them to pay their taxes for society when individually they’d be wealthier if they didn’t is a type of cooperation task. Asking people to volunteer while working together is a joint coordination-cooperation task. Tackling pollution through individual lifestyle changes and legal changes is a type of group task.
Democracy itself is a cooperation task utilizing wisdom of the crowd as long as all parties have equal say, though that say may take the form of competing lobbying groups. What matters is that some groups don’t get greater say than others or established dominant groups don’t get to suppress others.
Denying political groups you don’t like their freedom of speech, disrupting their events or trying to outlaw them entirely undermines democracy and the problem solving ability of the crowd. And the same is true if the experts or activists dominate the floor and change the group's outcome, always for the worse.