New study improves 'crowd wisdom' estimates

New study improves 'crowd wisdom' estimates
Aaron Leventman estimates how many beads are in the jar. Credit: Scott Wagner via Santa Fe Institute

In 1907, a statistician named Francis Galton recorded the entries from a weight-judging competition as people guessed the weight of an ox. Galton analyzed hundreds of estimates and found that while individual guesses varied wildly, the median of the entries was surprisingly accurate and within one percent of the ox's real weight. When Galton published his results, he ushered the theory of collective intelligence, or the "wisdom of crowds," into the public conscience.

Collective has its limits, though. In a new study published in the Journal of the Royal Society Interface, researchers Albert Kao (Harvard University), Andrew Berdahl (Santa Fe Institute), and their colleagues examined just how accurate our is and how individual bias and information sharing skew aggregate estimates. Using their findings, they developed a mathematical correction that takes into account bias and social information to generate an improved crowd estimate. In the study, their corrected measures were more accurate than the mean, median, and other traditional statistics.

"There is growing evidence that the wisdom of crowds can be really powerful," Kao says. "A lot of studies show that you can calculate the average of estimates and that average can be surprisingly good."

"However," adds Berdahl, "there is a great deal of evidence that people have strong biases in estimation and decision tasks."

The researchers recruited over 800 volunteers to participate in the study and asked each participant to guess the number of gumballs in a jar, which ranged over several orders of magnitude from 54 to more than 27,000. Additionally, they quantified how individuals incorporate social information into their own opinion. To do so, the researchers offered participants fake details about other people's guesses and allowed them to change their estimate in light of that information.

New study improves 'crowd wisdom' estimates
Participants in the study were asked to estimate how many gumballs or beads filled a given jar. (Gumballs pictured: 659. Beads pictured: 27,852) Credit: Albert Kao and Andrew Berdahl

Kao's team found that while estimates varied considerably, they were highly predictable. People tended to guess numbers smaller than the actual value and guessed a wider range of numbers for larger jars. Social also plays a role in collective wisdom. For example, the simulated revealed that peer advice more strongly influenced an individual if the knowledge suggested the actual number of items was higher than the guesser's initial estimate. Smaller guesses, even if more accurate, appear to be more frequently discounted.

The findings define a clear set of rules for collective estimation, which in turn provide insight into why we observe the wisdom of crowds. This understanding can improve group wisdom in a variety of settings. Whether guessing the weight of a fat ox or predicting the outcome of future elections, Kao and his colleagues have helped the grow smarter.

More information: Counteracting estimation bias and social influence to improve the wisdom of crowds, Journal of the Royal Society Interface (2018). rsif.royalsocietypublishing.or … .1098/rsif.2018.0130

Provided by Santa Fe Institute

Citation: New study improves 'crowd wisdom' estimates (2018, April 18) retrieved 14 April 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Averaging the wisdom of crowds


Feedback to editors