Crowd mentality

Sep 21, 2011 By Heather Wuebker
“As a group, people are collectively more intelligent than you might think,” says professor Mark Steyvers. “We all possess knowledge about different things, some more than others. If we pool these pieces of knowledge together, we can get a pretty accurate look at the big picture.” Credit: / Francesco Santalucia

Do you know what the price of gas will be in six months? How about the extent of the expected U.S. troop drawdown in Afghanistan by year’s end? As an individual, predicting such things with any degree of confidence may seem the stuff of science fiction, but UC Irvine cognitive scientists say it’s possible.

“As a group, people are collectively more intelligent than you might think,” says professor Mark Steyvers. “We all possess knowledge about different things, some more than others. If we pool these pieces of knowledge together, we can get a pretty accurate look at the big picture.”

It’s called the “” effect, and researchers are hoping to get your help putting it to the test.

Steyvers and fellow UCI cognitive sciences professors Michael Lee and William Batchelder are part of a national team given a grant by the Intelligence Advanced Research Projects Activity to develop statistical models based on the crowds concept that can, with increasing precision, forecast the future.

Working with the UCI trio are scientists from the University of Maryland, the University of Michigan, The Ohio State University, Fordham University, Wake Forest University, Wichita State University and private company Applied Research Associates.

They constitute one of five national teams to be funded by IARPA and must meet yearly goals to continue receiving support from the government agency.

In July, the UCI/multi-university team launched its models via a software program called Forecasting Ace. Through volunteer participation, the program will collect individual opinions on the likelihood of certain events within a specified time frame.

Before providing input, contributors must complete a short questionnaire in which they rate their subject-matter expertise in areas including science and technology, business and the economy, politics and policy, military and security, and sports and health.

After the self-assessment, participants can choose which questions about the future they’d like to answer. The software then aggregates and analyzes the collected data using models of human decision making created by the UCI scientists.

“Finding wisdom in crowds is fundamentally a cognitive science problem, because it’s about how people acquire knowledge and make judgments,” says Lee. The concept, while relatively recent, is similar to cultural consensus theory, which Batchelder has been developing for 30 years.

The newer computer and cognitive aspects draw on the expertise of Lee and Steyvers, who have been perfecting these modeling methods at UCI over the past four years through numerous lab experiments and analyses of real-world behavioral data. The professors, along with two student researchers, won a best-paper prize for this work at the July conference of the Cognitive Science Society.

The qualitative and quantitative results from these various studies have helped them construct novel ways of identifying experts by combining self-reported expertise levels and behavioral responses in their models.

As the time frame for a forecasting challenge – such as the likelihood of federal legislators agreeing to raise the U.S. debt ceiling by Aug. 2 – draws near, the UCI team runs the collected data through the models. During this process, “ace analysts” – true experts in their field – are identified, and their opinions are appropriately weighted, helping researchers derive a relatively sound prediction of the upcoming event’s outcome.

They currently don’t know a prediction’s degree of accuracy until the event does or does not occur, but continued data aggregation lets them hone the models to account for more complex and precise forecasting.

The resulting applied project has implications for government agencies interested in improving the accuracy of intelligence analysis. It may also have applications in the fields of business, medicine and policy.

Funding for the endeavor began in May and runs through April 2012, at which time it may be extended. During that period, the research program will strive to achieve a 20 percent increase in forecast accuracy compared with state-of-the-art alternatives. Subsequent years will aim for growth of 35, 50 and more percentage points.

Explore further: Understanding the economics of human trafficking

add to favorites email to friend print save as pdf

Related Stories

Now you see him

Mar 08, 2011

Imagine if Harry Potter’s cloak were real , or that you could blot out the sight of something as easily as pressing a mute button to eliminate sound. To some, this seems like “pi in the sky” ...

Consumer confidence hits five-year high in Michigan

Oct 27, 2010

( -- Despite Michigan’s continued economic malaise, residents’ optimism about the future is at its highest in nearly five years, according to Michigan State University’s latest State of the State ...

Recommended for you

Local education politics 'far from dead'

4 hours ago

Teach for America, known for recruiting teachers, is also setting its sights on capturing school board seats across the nation. Surprisingly, however, political candidates from the program aren't just pushing ...

First grade reading suffers in segregated schools

4 hours ago

A groundbreaking study from the Frank Porter Graham Child Development Institute (FPG) has found that African-American students in first grade experience smaller gains in reading when they attend segregated schools—but the ...

Why aren't consumers buying remanufactured products?

7 hours ago

Firms looking to increase market share of remanufactured consumer products will have to overcome a big barrier to do so, according to a recent study from the Penn State Smeal College of Business. Findings from faculty members ...

Expecting to teach enhances learning, recall

7 hours ago

People learn better and recall more when given the impression that they will soon have to teach newly acquired material to someone else, suggests new research from Washington University in St. Louis.

Understanding the economics of human trafficking

Jul 28, 2014

Although Europe is one of the strictest regions in the world when it comes to guaranteeing the respect of human rights, the number of people trafficked to or within the EU still amounts to several hundred ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Sep 21, 2011
The wisdom of crowds only works when other people do not know what others are thinking. Otherwise they start to align and make systematic mistakes en masse. See: Herd mentality or lemmings.