New study improves 'crowd wisdom' estimates

April 18, 2018, Santa Fe Institute
Aaron Leventman estimates how many beads are in the jar. Credit: Scott Wagner via Santa Fe Institute

In 1907, a statistician named Francis Galton recorded the entries from a weight-judging competition as people guessed the weight of an ox. Galton analyzed hundreds of estimates and found that while individual guesses varied wildly, the median of the entries was surprisingly accurate and within one percent of the ox's real weight. When Galton published his results, he ushered the theory of collective intelligence, or the "wisdom of crowds," into the public conscience.

Collective has its limits, though. In a new study published in the Journal of the Royal Society Interface, researchers Albert Kao (Harvard University), Andrew Berdahl (Santa Fe Institute), and their colleagues examined just how accurate our is and how individual bias and information sharing skew aggregate estimates. Using their findings, they developed a mathematical correction that takes into account bias and social information to generate an improved crowd estimate. In the study, their corrected measures were more accurate than the mean, median, and other traditional statistics.

"There is growing evidence that the wisdom of crowds can be really powerful," Kao says. "A lot of studies show that you can calculate the average of estimates and that average can be surprisingly good."

"However," adds Berdahl, "there is a great deal of evidence that people have strong biases in estimation and decision tasks."

The researchers recruited over 800 volunteers to participate in the study and asked each participant to guess the number of gumballs in a jar, which ranged over several orders of magnitude from 54 to more than 27,000. Additionally, they quantified how individuals incorporate social information into their own opinion. To do so, the researchers offered participants fake details about other people's guesses and allowed them to change their estimate in light of that information.

Participants in the study were asked to estimate how many gumballs or beads filled a given jar. (Gumballs pictured: 659. Beads pictured: 27,852) Credit: Albert Kao and Andrew Berdahl

Kao's team found that while estimates varied considerably, they were highly predictable. People tended to guess numbers smaller than the actual value and guessed a wider range of numbers for larger jars. Social also plays a role in collective wisdom. For example, the simulated revealed that peer advice more strongly influenced an individual if the knowledge suggested the actual number of items was higher than the guesser's initial estimate. Smaller guesses, even if more accurate, appear to be more frequently discounted.

The findings define a clear set of rules for collective estimation, which in turn provide insight into why we observe the wisdom of crowds. This understanding can improve group wisdom in a variety of settings. Whether guessing the weight of a fat ox or predicting the outcome of future elections, Kao and his colleagues have helped the grow smarter.

Explore further: Averaging the wisdom of crowds

More information: Counteracting estimation bias and social influence to improve the wisdom of crowds, Journal of the Royal Society Interface (2018). rsif.royalsocietypublishing.or … .1098/rsif.2018.0130

Related Stories

Averaging the wisdom of crowds

December 12, 2017

The best decisions are made on the basis of the average of various estimates, as confirmed by the research of Dennie van Dolder and Martijn van den Assem, scientists at VU Amsterdam. Using data from Holland Casino promotional ...

Crowds within crowd found to outperform 'wisdom of the crowd'

January 18, 2018

A team of researchers affiliated with institutions in Argentina, the U.S. and Germany has found that there is a way to improve on the "wisdom of the crowd"—separate the people in a given crowd into smaller groups and let ...

Recommended for you

Multiple stellar populations detected in the cluster Hodge 6

February 18, 2019

Using ESO's Very Large Telescope (VLT), astronomers have found that the cluster Hodge 6 hosts multiple stellar populations. The detection could provide important hints on the formation and evolution of Hodge 6 and star clusters ...

Predicting sequence from structure

February 18, 2019

One way to probe intricate biological systems is to block their components from interacting and see what happens. This method allows researchers to better understand cellular processes and functions, augmenting everyday laboratory ...

Energetic particles can bombard exoplanets

February 18, 2019

TRAPPIST-1 is a system of seven Earth-sized worlds orbiting an ultra-cool dwarf star about 120 light-years away. The star, and hence its system of planets, is thought to be between five-to-ten billion years old, up to twice ...

Meteorite source in asteroid belt not a single debris field

February 17, 2019

A new study published online in Meteoritics and Planetary Science finds that our most common meteorites, those known as L chondrites, come from at least two different debris fields in the asteroid belt. The belt contains ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.