Crowd mentality

Sep 21, 2011 By Heather Wuebker
“As a group, people are collectively more intelligent than you might think,” says professor Mark Steyvers. “We all possess knowledge about different things, some more than others. If we pool these pieces of knowledge together, we can get a pretty accurate look at the big picture.” Credit: iStockphoto.com / Francesco Santalucia

Do you know what the price of gas will be in six months? How about the extent of the expected U.S. troop drawdown in Afghanistan by year’s end? As an individual, predicting such things with any degree of confidence may seem the stuff of science fiction, but UC Irvine cognitive scientists say it’s possible.

“As a group, people are collectively more intelligent than you might think,” says professor Mark Steyvers. “We all possess knowledge about different things, some more than others. If we pool these pieces of knowledge together, we can get a pretty accurate look at the big picture.”

It’s called the “” effect, and researchers are hoping to get your help putting it to the test.

Steyvers and fellow UCI cognitive sciences professors Michael Lee and William Batchelder are part of a national team given a grant by the Intelligence Advanced Research Projects Activity to develop statistical models based on the crowds concept that can, with increasing precision, forecast the future.

Working with the UCI trio are scientists from the University of Maryland, the University of Michigan, The Ohio State University, Fordham University, Wake Forest University, Wichita State University and private company Applied Research Associates.

They constitute one of five national teams to be funded by IARPA and must meet yearly goals to continue receiving support from the government agency.

In July, the UCI/multi-university team launched its models via a software program called Forecasting Ace. Through volunteer participation, the program will collect individual opinions on the likelihood of certain events within a specified time frame.

Before providing input, contributors must complete a short questionnaire in which they rate their subject-matter expertise in areas including science and technology, business and the economy, politics and policy, military and security, and sports and health.

After the self-assessment, participants can choose which questions about the future they’d like to answer. The software then aggregates and analyzes the collected data using models of human decision making created by the UCI scientists.

“Finding wisdom in crowds is fundamentally a cognitive science problem, because it’s about how people acquire knowledge and make judgments,” says Lee. The concept, while relatively recent, is similar to cultural consensus theory, which Batchelder has been developing for 30 years.

The newer computer and cognitive aspects draw on the expertise of Lee and Steyvers, who have been perfecting these modeling methods at UCI over the past four years through numerous lab experiments and analyses of real-world behavioral data. The professors, along with two student researchers, won a best-paper prize for this work at the July conference of the Cognitive Science Society.

The qualitative and quantitative results from these various studies have helped them construct novel ways of identifying experts by combining self-reported expertise levels and behavioral responses in their models.

As the time frame for a forecasting challenge – such as the likelihood of federal legislators agreeing to raise the U.S. debt ceiling by Aug. 2 – draws near, the UCI team runs the collected data through the models. During this process, “ace analysts” – true experts in their field – are identified, and their opinions are appropriately weighted, helping researchers derive a relatively sound prediction of the upcoming event’s outcome.

They currently don’t know a prediction’s degree of accuracy until the event does or does not occur, but continued data aggregation lets them hone the models to account for more complex and precise forecasting.

The resulting applied project has implications for government agencies interested in improving the accuracy of intelligence analysis. It may also have applications in the fields of business, medicine and policy.

Funding for the endeavor began in May and runs through April 2012, at which time it may be extended. During that period, the research program will strive to achieve a 20 percent increase in forecast accuracy compared with state-of-the-art alternatives. Subsequent years will aim for growth of 35, 50 and more percentage points.

Explore further: Why are UK teenagers skipping school?

add to favorites email to friend print save as pdf

Related Stories

Now you see him

Mar 08, 2011

Imagine if Harry Potter’s cloak were real , or that you could blot out the sight of something as easily as pressing a mute button to eliminate sound. To some, this seems like “pi in the sky” ...

Consumer confidence hits five-year high in Michigan

Oct 27, 2010

(PhysOrg.com) -- Despite Michigan’s continued economic malaise, residents’ optimism about the future is at its highest in nearly five years, according to Michigan State University’s latest State of the State ...

Recommended for you

Why are UK teenagers skipping school?

11 hours ago

Analysis of the results of a large-scale survey reveals the extent of truancy in English secondary schools and sheds light on the mental health of the country's teens.

Fewer lectures, more group work

11 hours ago

Professor Cees van der Vleuten from Maastricht University is a Visiting Professor at Wits University who believes that learning should be student centred.

How to teach all students to think critically

12 hours ago

All first year students at the University of Technology Sydney could soon be required to take a compulsory maths course in an attempt to give them some numerical thinking skills. ...

Consumer loyalty driven by aesthetics over functionality

Dec 17, 2014

When designing a new car, manufacturers might try to attract consumers with more horsepower, increased fuel efficiency or a lower price point. But new research from San Francisco State University shows consumers' loyalty ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

Corban
not rated yet Sep 21, 2011
The wisdom of crowds only works when other people do not know what others are thinking. Otherwise they start to align and make systematic mistakes en masse. See: Herd mentality or lemmings.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.